Allow threads to be marked as unsuspendable by kForUserCode

There was a possible deadlock between jit-threads being suspended and
another thread waiting for the jit-thread to finish their work. If the
jit-thread hit a suspend-point the process would deadlock. This fixes
this by allowing threads to be marked as unsuspendable by user-code.
This prevents the issue by marking jit and gc threads as unsuspendable
by user code. Agents attempting to suspend them using JVMTI will
succeed (and see the thread as suspended) but internal runtime methods
will not see the thread as suspended and the thread will not be
prevented from moving into the kRunnable state. A thread that is
unsuspendable trying to suspend itself using JVMTI will get
ERR(INTERNAL) and a log message.

Doing this requires that we rewrite the JVMTI thread suspension code
somewhat so it will now perform an unconditional kInternal suspension
prior to trying to suspend the thread kForUserCode. The kInternal
suspension is then lifted. This ensures that everything is done
atomically even if the kForUserCode won't stop the thread.

Test: ./test.py --host
Test: ./art/tools/run-libjdwp-tests.sh --mode=host
Test: ./art/tools/run-libjdwp-tests.sh \
        --mode=host                    \
        --variant=x64                  \
        --test org.apache.harmony.jpda.tests.jdwp.EventModifiers.InstanceOnlyModifierTest

Bug: 70838465
Bug: 111348762

Change-Id: I91211641b82416664bf5abd8546efebf4f672f12
diff --git a/runtime/thread.h b/runtime/thread.h
index edc429b..d169a62 100644
--- a/runtime/thread.h
+++ b/runtime/thread.h
@@ -989,6 +989,17 @@
     --tls32_.disable_thread_flip_count;
   }
 
+  // Returns true if the thread is subject to user_code_suspensions.
+  bool CanBeSuspendedByUserCode() const {
+    return can_be_suspended_by_user_code_;
+  }
+
+  // Sets CanBeSuspenededByUserCode and adjusts the suspend-count as needed. This may only be called
+  // when running on the current thread. It is **absolutely required** that this be called only on
+  // the Thread::Current() thread.
+  void SetCanBeSuspendedByUserCode(bool can_be_suspended_by_user_code)
+      REQUIRES(!Locks::thread_suspend_count_lock_, !Locks::user_code_suspension_lock_);
+
   // Returns true if the thread is allowed to call into java.
   bool CanCallIntoJava() const {
     return can_call_into_java_;
@@ -1552,8 +1563,9 @@
     // critical section enter.
     uint32_t disable_thread_flip_count;
 
-    // How much of 'suspend_count_' is by request of user code, used to distinguish threads
-    // suspended by the runtime from those suspended by user code.
+    // If CanBeSuspendedByUserCode, how much of 'suspend_count_' is by request of user code, used to
+    // distinguish threads suspended by the runtime from those suspended by user code. Otherwise
+    // this is just a count of how many user-code suspends have been attempted (but were ignored).
     // This should have GUARDED_BY(Locks::user_code_suspension_lock_) but auto analysis cannot be
     // told that AssertHeld should be good enough.
     int user_code_suspend_count GUARDED_BY(Locks::thread_suspend_count_lock_);
@@ -1772,6 +1784,10 @@
   // By default this is true.
   bool can_call_into_java_;
 
+  // True if the thread is subject to user-code suspension. By default this is true. This can only
+  // be false for threads where '!can_call_into_java_'.
+  bool can_be_suspended_by_user_code_;
+
   friend class Dbg;  // For SetStateUnsafe.
   friend class gc::collector::SemiSpace;  // For getting stack traces.
   friend class Runtime;  // For CreatePeer.