session-android/app/src/main/java/org/thoughtcrime/securesms/webrtc/audio/SignalBluetoothManager.kt

365 lines
12 KiB
Kotlin
Raw Normal View History

Add one on one calls over clearnet (#864) * feat: adding basic webrtc deps and test activity * more testing code * feat: add protos and bump version * feat: added basic call functionality * feat: adding UI and flipping cameras * feat: add stats and starting call bottom sheet * feat: hanging up and bottom sheet behaviors should work now * feat: add call stats report on frontend * feat: add relay toggle for answer and offer * fix: add keep screen on and more end call message on back pressed / on finish * refactor: removing and replacing dagger 1 dep with android hilt * feat: include latest proto * feat: update to utilise call ID * feat: add stun and turn * refactor: playing around with deps and transport types * feat: adding call service functionality and permissions for calls * feat: add call manager and more static intent building functions for WebRtcCallService.kt * feat: adding ringers and more audio boilerplate * feat: audio manager call service boilerplate * feat: update kotlin and add in call view model and more management functions * refactor: moving call code around to service and viewmodel interactions * feat: plugging CallManager.kt into view model and service, fixing up dependencies * feat: implementing more WebRtcCallService.kt functions and handlers for actions as well as lifecycle * feat: adding more lifecycle vm and callmanager / call service functionality * feat: adding more command handlers in WebRtcCallService.kt * feat: more commands handled, adding lock manager and bluetooth permissions * feat: adding remainder of basic functionality to services and CallManager.kt * feat: hooking up calls and fixing broken dependencies and compile errors * fix: add timestamp to incoming call * feat: some connection and service launching / ring lifecycle * feat: call establishing and displaying * fix: fixing call connect flows * feat: ringers and better state handling * feat: updating call layout * feat: add fixes to bluetooth and begin the network renegotiation * feat: add call related permissions and more network handover tests * fix: don't display call option in conversation and don't show notification if option not enabled * fix: incoming ringer fix on receiving call, call notification priorities and notification channel update * build: update build number for testing * fix: bluetooth auto-connection and re-connection fixes, removing finished todos, allowing self-send call messages for deduping answers * feat: add pre-offer information and action handling in web rtc call service * refactor: discard offer messages from non-matching pre-offers we are already expecting * build: build numbers and version name update * feat: handle discarding pending calls from linked devices * feat: add signing props to release config build * docs: fix comment on time being 300s (5m) instead of 30s * feat: adding call messages for incoming/outgoing/missed * refactor: handle in-thread call notifications better and replace deny button intent with denyCallIntent instead of hangup * feat: add a hangup via data channel message * feat: process microphone enabled events and remove debuggable from build.gradle * feat: add first call notification * refactor: set the buttons to match iOS in terms of enable disable and colours * refactor: change the call logos in control messages * refactor: more bluetooth improvements * refactor: move start ringer and init of audio manager to CallManager.kt and string fix up * build: remove debuggable for release build * refactor: replace call icons * feat: adding a call time display * refactor: change the call time to update every second * refactor: testing out the full screen intents * refactor: wrapper use corrected session description, set title to recipient displayName, indicate session calls * fix: crash on view with a parent already attached * refactor: aspect ratio fit preserved * refactor: add wantsToAnswer ability in pre-init for fullscreenintent * refactor: prevent calls from non hasSent participants * build: update gradle code * refactor: replace timeout schedule with a seconds count * fix: various bug fixes for calls * fix: remove end call from busy * refactor: use answerCall instead of manual intent building again * build: new version * feat: add silenced notifications for call notification builder. check pre-offer and connecting state for pending connection * build: update build number * fix: text color uses overridden style value * fix: remove wrap content for renderers and look more at recovering from network switches * build: update build number * refactor: remove whitespace * build: update build number * refactor: used shared number for BatchMessageReceiveJob.kt parameter across pollers * fix: glide in update crash * fix: bug fixes for self-send answer / hangup messages * build: update build number * build: update build.gradle number * refactor: compile errors and refactoring to view binding * fix: set the content to binding.root view * build: increase build number * build: update build numbers * feat: adding base for rotation and picking random subset of turn servers * feat: starting the screen rotation processing * feat: setting up rotation for the remote render view * refactor: applying rotation and mirroring based on front / rear cameras that wraps nicely, only scale reworking needed * refactor: calls video stretching but consistent * refactor: state machine and tests for the transition events * feat: new call state processing * refactor: adding reconnecting logic and visuals * feat: state machine reconnect logic wip * feat: add reconnecting and merge fixes * feat: check new session based off current state * feat: reconnection logic works correctly now * refactor: reduce TIMEOUT_SECONDS to 30 from 90 * feat: reset peer connection on DC to prevent ICE messages from old connection or stale state in reconnecting * refactor: add null case * fix: set approved on new outgoing threads, use approved more deeply and invalidate the options menu on recipient modified. Add approvedMe flag toggles for visible message receive * fix: add name update in action bar on modified, change where approvedMe is set * build: increment build number * build: update build number * fix: merge compile errors and increment build number * refactor: remove negotiation based on which party dropped connection * refactor: call reconnection improvement tested cross platform to re-establish * refactor: failed and disconnect events only handled if either the reconnect or the timeout runnables are not set * build: update version number * fix: reduce timeout * fix: fixes the incoming hangup logic for linked devices * refactor: match iOS styling for call activity closer * chore: upgrade build numbers * feat: add in call settings dialog for if calls is disabled in conversation * feat: add a first call missed control message and info popup with link to privacy settings * fix: looking at crash for specific large transaction in NotificationManager * refactor: removing the people in case transaction size reduces to fix notif crash * fix: comment out the entire send multiple to see if it fixes the issue * refactor: revert to including the full notification process in a try/catch to handle weird responses from NotificationManager * fix: add in notification settings prompt for calls and try to fall back to dirty full screen intent / start activity if we're allowed * build: upgrade build number
2022-04-19 06:25:40 +02:00
package org.thoughtcrime.securesms.webrtc.audio
import android.Manifest
import android.bluetooth.BluetoothAdapter
import android.bluetooth.BluetoothDevice
import android.bluetooth.BluetoothHeadset
import android.bluetooth.BluetoothProfile
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.media.AudioManager
import org.session.libsignal.utilities.Log
import org.thoughtcrime.securesms.webrtc.AudioManagerCommand
import java.util.concurrent.TimeUnit
/**
* Manages the bluetooth lifecycle with a headset. This class doesn't make any
* determination on if bluetooth should be used. It determines if a device is connected,
* reports that to the [SignalAudioManager], and then handles connecting/disconnecting
* to the device if requested by [SignalAudioManager].
*/
class SignalBluetoothManager(
private val context: Context,
private val audioManager: SignalAudioManager,
private val androidAudioManager: AudioManagerCompat,
private val handler: SignalAudioHandler
) {
var state: State = State.UNINITIALIZED
get() {
handler.assertHandlerThread()
return field
}
private set
private var bluetoothAdapter: BluetoothAdapter? = null
private var bluetoothHeadset: BluetoothHeadset? = null
private var scoConnectionAttempts = 0
private val bluetoothListener = BluetoothServiceListener()
private var bluetoothReceiver: BluetoothHeadsetBroadcastReceiver? = null
private val bluetoothTimeout = { onBluetoothTimeout() }
fun start() {
handler.assertHandlerThread()
Log.d(TAG, "start(): $state")
if (state != State.UNINITIALIZED) {
Log.w(TAG, "Invalid starting state")
return
}
bluetoothHeadset = null
scoConnectionAttempts = 0
bluetoothAdapter = BluetoothAdapter.getDefaultAdapter()
if (bluetoothAdapter == null) {
Log.i(TAG, "Device does not support Bluetooth")
return
}
if (!androidAudioManager.isBluetoothScoAvailableOffCall) {
Log.w(TAG, "Bluetooth SCO audio is not available off call")
return
}
if (bluetoothAdapter?.getProfileProxy(context, bluetoothListener, BluetoothProfile.HEADSET) != true) {
Log.e(TAG, "BluetoothAdapter.getProfileProxy(HEADSET) failed")
return
}
val bluetoothHeadsetFilter = IntentFilter().apply {
addAction(BluetoothHeadset.ACTION_CONNECTION_STATE_CHANGED)
addAction(AudioManager.ACTION_SCO_AUDIO_STATE_UPDATED)
}
bluetoothReceiver = BluetoothHeadsetBroadcastReceiver()
context.registerReceiver(bluetoothReceiver, bluetoothHeadsetFilter)
Log.i(TAG, "Headset profile state: ${bluetoothAdapter?.getProfileConnectionState(BluetoothProfile.HEADSET)?.toStateString()}")
Log.i(TAG, "Bluetooth proxy for headset profile has started")
state = State.UNAVAILABLE
}
fun stop() {
handler.assertHandlerThread()
Log.d(TAG, "stop(): state: $state")
if (bluetoothAdapter == null) {
return
}
stopScoAudio()
if (state == State.UNINITIALIZED) {
return
}
bluetoothReceiver?.let { receiver ->
try {
context.unregisterReceiver(receiver)
} catch (e: Exception) {
Log.e(TAG,"error unregistering bluetoothReceiver", e)
}
}
bluetoothReceiver = null
cancelTimer()
bluetoothAdapter?.closeProfileProxy(BluetoothProfile.HEADSET, bluetoothHeadset)
bluetoothHeadset = null
bluetoothAdapter = null
state = State.UNINITIALIZED
}
fun startScoAudio(): Boolean {
handler.assertHandlerThread()
Log.i(TAG, "startScoAudio(): $state attempts: $scoConnectionAttempts")
if (scoConnectionAttempts >= MAX_CONNECTION_ATTEMPTS) {
Log.w(TAG, "SCO connection attempts maxed out")
return false
}
if (state != State.AVAILABLE) {
Log.w(TAG, "SCO connection failed as no headset available")
return false
}
state = State.CONNECTING
androidAudioManager.startBluetoothSco()
scoConnectionAttempts++
startTimer()
return true
}
fun stopScoAudio() {
handler.assertHandlerThread()
Log.i(TAG, "stopScoAudio(): $state")
if (state != State.CONNECTING && state != State.CONNECTED) {
return
}
cancelTimer()
androidAudioManager.stopBluetoothSco()
androidAudioManager.isBluetoothScoOn = false
state = State.DISCONNECTING
}
fun updateDevice() {
handler.assertHandlerThread()
Log.d(TAG, "updateDevice(): state: $state")
if (state == State.UNINITIALIZED || bluetoothHeadset == null) {
return
}
if (bluetoothAdapter!!.getProfileConnectionState(BluetoothProfile.HEADSET) !in arrayOf(BluetoothProfile.STATE_CONNECTED)) {
state = State.UNAVAILABLE
Log.i(TAG, "No connected bluetooth headset")
} else {
state = State.AVAILABLE
Log.i(TAG, "Connected bluetooth headset.")
}
}
private fun updateAudioDeviceState() {
audioManager.handleCommand(AudioManagerCommand.UpdateAudioDeviceState)
}
private fun startTimer() {
handler.postDelayed(bluetoothTimeout, SCO_TIMEOUT)
}
private fun cancelTimer() {
handler.removeCallbacks(bluetoothTimeout)
}
private fun onBluetoothTimeout() {
Log.i(TAG, "onBluetoothTimeout: state: $state bluetoothHeadset: $bluetoothHeadset")
if (state == State.UNINITIALIZED || bluetoothHeadset == null || state != State.CONNECTING) {
return
}
var scoConnected = false
if (audioManager.isBluetoothScoOn()) {
Log.d(TAG, "Connected with device")
scoConnected = true
} else {
Log.d(TAG, "Not connected with device")
}
if (scoConnected) {
Log.i(TAG, "Device actually connected and not timed out")
state = State.CONNECTED
scoConnectionAttempts = 0
} else {
Log.w(TAG, "Failed to connect after timeout")
stopScoAudio()
}
updateAudioDeviceState()
}
private fun onServiceConnected(proxy: BluetoothHeadset?) {
bluetoothHeadset = proxy
androidAudioManager.isBluetoothScoOn = true
updateAudioDeviceState()
}
private fun onServiceDisconnected() {
stopScoAudio()
bluetoothHeadset = null
state = State.UNAVAILABLE
updateAudioDeviceState()
}
private fun onHeadsetConnectionStateChanged(connectionState: Int) {
Log.i(TAG, "onHeadsetConnectionStateChanged: state: $state connectionState: ${connectionState.toStateString()}")
when (connectionState) {
BluetoothHeadset.STATE_CONNECTED -> {
scoConnectionAttempts = 0
updateAudioDeviceState()
}
BluetoothHeadset.STATE_DISCONNECTED -> {
stopScoAudio()
updateAudioDeviceState()
}
}
}
private fun onAudioStateChanged(audioState: Int, isInitialStateChange: Boolean) {
Log.i(TAG, "onAudioStateChanged: state: $state audioState: ${audioState.toStateString()} initialSticky: $isInitialStateChange")
if (audioState == AudioManager.SCO_AUDIO_STATE_CONNECTED) {
cancelTimer()
if (state == State.CONNECTING) {
Log.d(TAG, "Bluetooth audio SCO is now connected")
state = State.CONNECTED
scoConnectionAttempts = 0
updateAudioDeviceState()
} else {
Log.w(TAG, "Unexpected state ${audioState.toStateString()}")
}
} else if (audioState == AudioManager.SCO_AUDIO_STATE_CONNECTING) {
Log.d(TAG, "Bluetooth audio SCO is now connecting...")
} else if (audioState == AudioManager.SCO_AUDIO_STATE_DISCONNECTED) {
Log.d(TAG, "Bluetooth audio SCO is now disconnected")
if (isInitialStateChange) {
Log.d(TAG, "Ignore ${audioState.toStateString()} initial sticky broadcast.")
return
}
updateAudioDeviceState()
}
}
private inner class BluetoothServiceListener : BluetoothProfile.ServiceListener {
override fun onServiceConnected(profile: Int, proxy: BluetoothProfile?) {
if (profile == BluetoothProfile.HEADSET) {
handler.post {
if (state != State.UNINITIALIZED) {
onServiceConnected(proxy as? BluetoothHeadset)
}
}
}
}
override fun onServiceDisconnected(profile: Int) {
if (profile == BluetoothProfile.HEADSET) {
handler.post {
if (state != State.UNINITIALIZED) {
onServiceDisconnected()
}
}
}
}
}
private inner class BluetoothHeadsetBroadcastReceiver : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) {
if (intent.action == BluetoothHeadset.ACTION_CONNECTION_STATE_CHANGED) {
val connectionState: Int = intent.getIntExtra(BluetoothHeadset.EXTRA_STATE, BluetoothHeadset.STATE_DISCONNECTED)
handler.post {
if (state != State.UNINITIALIZED) {
onHeadsetConnectionStateChanged(connectionState)
}
}
} else if (intent.action == BluetoothHeadset.ACTION_AUDIO_STATE_CHANGED) {
// val connectionState: Int = intent.getIntExtra(BluetoothHeadset.EXTRA_STATE, BluetoothHeadset.STATE_AUDIO_DISCONNECTED)
// handler.post {
// if (state != State.UNINITIALIZED) {
// onAudioStateChanged(connectionState, isInitialStickyBroadcast)
// }
// }
} else if (intent.action == AudioManager.ACTION_SCO_AUDIO_STATE_UPDATED) {
val scoState: Int = intent.getIntExtra(AudioManager.EXTRA_SCO_AUDIO_STATE, AudioManager.ERROR)
handler.post {
if (state != State.UNINITIALIZED) {
onAudioStateChanged(scoState, isInitialStickyBroadcast)
}
}
}
}
}
enum class State {
UNINITIALIZED,
UNAVAILABLE,
AVAILABLE,
DISCONNECTING,
CONNECTING,
CONNECTED,
ERROR;
fun shouldUpdate(): Boolean {
return this == AVAILABLE || this == UNAVAILABLE || this == DISCONNECTING
}
fun hasDevice(): Boolean {
return this == CONNECTED || this == CONNECTING || this == AVAILABLE
}
}
companion object {
private val TAG = Log.tag(SignalBluetoothManager::class.java)
private val SCO_TIMEOUT = TimeUnit.SECONDS.toMillis(4)
private const val MAX_CONNECTION_ATTEMPTS = 2
}
}
private fun Int.toStateString(): String {
return when (this) {
BluetoothAdapter.STATE_DISCONNECTED -> "DISCONNECTED"
BluetoothAdapter.STATE_CONNECTED -> "CONNECTED"
BluetoothAdapter.STATE_CONNECTING -> "CONNECTING"
BluetoothAdapter.STATE_DISCONNECTING -> "DISCONNECTING"
BluetoothAdapter.STATE_OFF -> "OFF"
BluetoothAdapter.STATE_ON -> "ON"
BluetoothAdapter.STATE_TURNING_OFF -> "TURNING_OFF"
BluetoothAdapter.STATE_TURNING_ON -> "TURNING_ON"
else -> "UNKNOWN"
}
}
private fun Int.toScoString(): String = when (this) {
AudioManager.SCO_AUDIO_STATE_DISCONNECTED -> "DISCONNECTED"
AudioManager.SCO_AUDIO_STATE_CONNECTED -> "CONNECTED"
AudioManager.SCO_AUDIO_STATE_CONNECTING -> "CONNECTING"
AudioManager.SCO_AUDIO_STATE_ERROR -> "ERROR"
else -> "UNKNOWN"
}