ASoC: SOF: core: fix null-ptr-deref bug during device removal

The DSP should be notified for device removal only if the
probe was successful. Fixes the following KASAN bug:
BUG: KASAN: null-ptr-deref in sof_ipc_tx_message+0x80/0x160 [snd_sof]

Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Reviewed-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Link: https://lore.kernel.org/r/20200707204027.114169-1-pierre-louis.bossart@linux.intel.com
Signed-off-by: Mark Brown <broonie@kernel.org>
diff --git a/sound/soc/sof/core.c b/sound/soc/sof/core.c
index 339c493..adc7c37 100644
--- a/sound/soc/sof/core.c
+++ b/sound/soc/sof/core.c
@@ -345,15 +345,15 @@ int snd_sof_device_remove(struct device *dev)
 	struct snd_sof_pdata *pdata = sdev->pdata;
 	int ret;
 
-	ret = snd_sof_dsp_power_down_notify(sdev);
-	if (ret < 0)
-		dev_warn(dev, "error: %d failed to prepare DSP for device removal",
-			 ret);
-
 	if (IS_ENABLED(CONFIG_SND_SOC_SOF_PROBE_WORK_QUEUE))
 		cancel_work_sync(&sdev->probe_work);
 
 	if (sdev->fw_state > SOF_FW_BOOT_NOT_STARTED) {
+		ret = snd_sof_dsp_power_down_notify(sdev);
+		if (ret < 0)
+			dev_warn(dev, "error: %d failed to prepare DSP for device removal",
+				 ret);
+
 		snd_sof_fw_unload(sdev);
 		snd_sof_ipc_free(sdev);
 		snd_sof_free_debug(sdev);