Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import { useEffect, useRef } from 'react'
import { useRemoteParticipants, useRoomContext } from '@livekit/components-react'
import { RemoteParticipant, RemoteTrackPublication, Track } from 'livekit-client'
import { useSnapshot } from 'valtio'
import { userChoicesStore } from '@/stores/userChoices'

const TARGET_LEVEL = 0.12
const ADAPTATION_RATE = 0.05
const MIN_GAIN = 0.2
const MAX_GAIN = 3.0
const TICK_INTERVAL_MS = 500

interface ParticipantGainState {
currentGain: number
smoothedLevel: number
}

/**
* Adaptive per-participant audio level normalization (issue #1345).
*
* Every TICK_INTERVAL_MS, samples each remote participant's audioLevel,
* smooths it via EMA, and gradually adjusts their audio element volume
* toward TARGET_LEVEL. Gain is clamped to [MIN_GAIN, MAX_GAIN].
* Resets all volumes to 1.0 when disabled.
*/
export const useAudioLevelEqualization = () => {
const room = useRoomContext()
const remoteParticipants = useRemoteParticipants()
const { audioLevelEqualizationEnabled } = useSnapshot(userChoicesStore)
const gainStateRef = useRef<Map<string, ParticipantGainState>>(new Map())

useEffect(() => {
if (!audioLevelEqualizationEnabled) {
for (const participant of room.remoteParticipants.values()) {
const audioElement = getAudioElement(participant)
if (audioElement) audioElement.volume = 1.0
}
gainStateRef.current.clear()
return
}

const tick = () => {
for (const participant of room.remoteParticipants.values()) {
const sid = participant.sid
const audioLevel = participant.audioLevel ?? 0

if (!gainStateRef.current.has(sid)) {
gainStateRef.current.set(sid, { currentGain: 1.0, smoothedLevel: audioLevel })
}

const state = gainStateRef.current.get(sid)!

// EMA to avoid reacting to transient spikes
state.smoothedLevel =
state.smoothedLevel * (1 - ADAPTATION_RATE) + audioLevel * ADAPTATION_RATE

// Only adjust gain when participant is speaking, not on silence/noise floor
if (state.smoothedLevel > 0.01) {
Comment on lines +55 to +58

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

4. Pauses make the next words too loud 🐞 Bug ≡ Correctness

smoothedLevel continues incorporating zero samples, while the speaking guard tests that slowly
decaying average rather than the current sample. After someone stops speaking the state remains
above 0.01 for many 500 ms ticks and keeps increasing the gain, so their next utterance starts
over-amplified.
Agent Prompt
## Issue description
Silence is folded into the level average, but the previous average is then treated as evidence that the participant is still speaking. This drives gain upward during ordinary pauses.

## Issue Context
Gate adaptation using a current speaking signal or current raw level, and retain or initialize the speaking-level estimate separately so silence does not lower the reference used to calculate gain.

## Fix Focus Areas
- src/frontend/src/features/rooms/livekit/hooks/useAudioLevelEqualization.ts[42-62]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

const ratio = TARGET_LEVEL / state.smoothedLevel
state.currentGain =
state.currentGain * (1 - ADAPTATION_RATE) + ratio * ADAPTATION_RATE
}

state.currentGain = Math.min(MAX_GAIN, Math.max(MIN_GAIN, state.currentGain))

const audioElement = getAudioElement(participant)
if (audioElement) audioElement.volume = state.currentGain
Comment on lines +64 to +67

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Quiet speakers halt audio leveling 📎 Requirement gap ≡ Correctness

tick clamps state.currentGain at 3.0 and writes it directly to HTMLMediaElement.volume,
whose valid range ends at 1.0. Whenever a participant’s sampled level is below the 0.12 target,
the calculated ratio can exceed one—for example, a level of 0.02 yields 1.25 on the first
update—so the assignment throws, prevents that quiet speaker from being boosted, and aborts the
interval callback before later participants are adjusted.
Agent Prompt
## Issue description
The equalizer writes gain values as high as `3.0` to `HTMLMediaElement.volume`, which only accepts values from `0` through `1`. This causes a range error instead of amplifying quiet participants and can stop the current interval callback before remaining participants are processed.

## Issue Context
The algorithm intentionally supports gains through `3.0`, so amplification requires an audio path that accepts gain above unity, such as a Web Audio `GainNode`. Tie the graph lifecycle and cleanup to each remote track, and ensure one participant’s gain failure cannot prevent processing the remaining participants. Do not merely clamp the value at `1.0`, because that would still fail the quiet-speaker use case.

## Fix Focus Areas
- src/frontend/src/features/rooms/livekit/hooks/useAudioLevelEqualization.ts[7-10]
- src/frontend/src/features/rooms/livekit/hooks/useAudioLevelEqualization.ts[58-67]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

}

// Clean up state for departed participants
for (const sid of gainStateRef.current.keys()) {
if (!room.remoteParticipants.has(sid)) {
gainStateRef.current.delete(sid)
}
}
}

const intervalId = setInterval(tick, TICK_INTERVAL_MS)
return () => clearInterval(intervalId)
}, [audioLevelEqualizationEnabled, room, remoteParticipants])
}

function getAudioElement(participant: RemoteParticipant): HTMLAudioElement | null {
const micPub = participant.getTrackPublication(
Track.Source.Microphone
) as RemoteTrackPublication | undefined

if (!micPub?.trackSid) return null

return document.querySelector<HTMLAudioElement>(
`audio[data-lk-sid="${micPub.trackSid}"]`
)
Comment on lines +90 to +92

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

3. Equalization never reaches remote audio 🐞 Bug ≡ Correctness

getAudioElement searches for an audio[data-lk-sid] node, but remote playback is delegated to
RoomAudioRenderer and no repository-owned renderer creates an audio element with that attribute.
Every tick therefore receives null and skips the only volume assignment, so enabling the setting
has no effect for remote participants.
Agent Prompt
## Issue description
Audio equalization searches for a `data-lk-sid` audio element that the current remote-audio rendering path does not expose, leaving the feature ineffective.

## Issue Context
Remote audio is rendered through LiveKit's `RoomAudioRenderer`. Integrate equalization with the actual rendered track or element rather than relying on an unsupported DOM attribute.

## Fix Focus Areas
- src/frontend/src/features/rooms/livekit/hooks/useAudioLevelEqualization.ts[83-92]
- src/frontend/src/features/rooms/livekit/prefabs/VideoConference.tsx[143-155]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

}
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import { isFireFox } from '@/utils/livekit'
import { MediaStateObserver } from '../components/MediaStateObserver'
import { RoomMetadataSynchronizer } from '../components/RoomMetadataSynchronizer'
import { useNoiseReduction } from '../hooks/useNoiseReduction'
import { useAudioLevelEqualization } from '../hooks/useAudioLevelEqualization'
import { VideoResolutionSubscription } from '../components/VideoResolutionSubscription'
import { SettingsDialogProvider } from '@/features/settings/components/SettingsDialogProvider'
import { MuteAlertDialogProvider } from '@/features/rooms/livekit/components/MuteAlertDialogProvider'
Expand Down Expand Up @@ -75,6 +76,7 @@ const getScreenSharePermissionDeniedScope = (
*/
export function VideoConference({ ...props }: VideoConferenceProps) {
useNoiseReduction()
useAudioLevelEqualization()

const { isOpen: isPictureInPictureOpen } = usePictureInPicture()

Expand Down
19 changes: 18 additions & 1 deletion src/frontend/src/features/settings/components/tabs/AudioTab.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
saveAudioInputDeviceId,
saveAudioOutputDeviceId,
saveNoiseReductionEnabled,
saveAudioLevelEqualizationEnabled,
userChoicesStore,
} from '@/stores/userChoices'
import { captureEvent } from '@/features/analytics/telemetry'
Expand All @@ -30,7 +31,7 @@ export const AudioTab = ({ id }: AudioTabProps) => {
const { t } = useTranslation('settings')
const { localParticipant } = useRoomContext()

const { noiseReductionEnabled, audioDeviceId, audioOutputDeviceId } =
const { noiseReductionEnabled, audioDeviceId, audioOutputDeviceId, audioLevelEqualizationEnabled } =
useSnapshot(userChoicesStore)

const isSpeaking = useIsSpeaking(localParticipant)
Expand Down Expand Up @@ -136,6 +137,22 @@ export const AudioTab = ({ id }: AudioTabProps) => {
<div />
</RowWrapper>
)}
{/* Audio level equalization — issue #1345 */}
<RowWrapper heading={t('audio.audioLevelEqualization.heading')} beta>
<Switch
aria-label={t(
`audio.audioLevelEqualization.ariaLabel.${audioLevelEqualizationEnabled ? 'disable' : 'enable'}`
)}
isSelected={audioLevelEqualizationEnabled}
onChange={(v) => {
saveAudioLevelEqualizationEnabled(v)
if (v) captureEvent('audio-level-equalization-init')
}}
>
{t('audio.audioLevelEqualization.label')}
</Switch>
<div />
Comment on lines +152 to +154

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Users miss the setting explanation 📎 Requirement gap ≡ Correctness

AudioTab renders an empty <div /> beside the switch instead of the translated
audio.audioLevelEqualization.description value. Whenever users evaluate this beta setting, they
see its label but not the available explanation of its continuous per-participant behavior.
Agent Prompt
## Issue description
The audio-leveling control omits the required one-line explanation even though translated description text was added.

## Issue Context
Render the description alongside the toggle using the settings tab's established text styling and translation function.

## Fix Focus Areas
- src/frontend/src/features/settings/components/tabs/AudioTab.tsx[140-155]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

</RowWrapper>
</TabPanel>
)
}
9 changes: 9 additions & 0 deletions src/frontend/src/locales/de/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,15 @@
"disable": "Geräuschunterdrückung deaktivieren"
}
},
"audioLevelEqualization": {
"label": "Lautstärke der Teilnehmer automatisch angleichen",
"heading": "Audio-Pegelanpassung",
"description": "Passt die Lautstärke jedes Teilnehmers kontinuierlich an, damit alle gleich laut zu hören sind.",
"ariaLabel": {
"enable": "Audio-Pegelanpassung aktivieren",
"disable": "Audio-Pegelanpassung deaktivieren"
}
},
"speakers": {
"heading": "Lautsprecher",
"label": "Audioausgabe wählen",
Expand Down
9 changes: 9 additions & 0 deletions src/frontend/src/locales/en/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,15 @@
"disable": "Disable noise reduction"
}
},
"audioLevelEqualization": {
"label": "Automatically balance participant audio levels",
"heading": "Audio level equalization",
"description": "Continuously adjusts each participant's volume so everyone sounds equally loud.",
"ariaLabel": {
"enable": "Enable audio level equalization",
"disable": "Disable audio level equalization"
}
},
"speakers": {
"heading": "Speakers",
"label": "Select your audio output",
Expand Down
9 changes: 9 additions & 0 deletions src/frontend/src/locales/fr/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,15 @@
"disable": "Désactiver la réduction du bruit"
}
},
"audioLevelEqualization": {
"label": "Équilibrer automatiquement les niveaux audio des participants",
"heading": "Égalisation du niveau audio",
"description": "Ajuste continuellement le volume de chaque participant pour que tout le monde soit entendu à égalité.",
"ariaLabel": {
"enable": "Activer l'égalisation du niveau audio",
"disable": "Désactiver l'égalisation du niveau audio"
}
},
"speakers": {
"heading": "Haut-parleurs",
"label": "Sélectionner votre sortie audio",
Expand Down
9 changes: 9 additions & 0 deletions src/frontend/src/locales/nl/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,15 @@
"disable": "Ruisonderdrukking uitschakelen"
}
},
"audioLevelEqualization": {
"label": "Audioniveaus van deelnemers automatisch balanceren",
"heading": "Audioniveauegalisatie",
"description": "Past het volume van elke deelnemer continu aan zodat iedereen even goed te horen is.",
"ariaLabel": {
"enable": "Audioniveauegalisatie inschakelen",
"disable": "Audioniveauegalisatie uitschakelen"
}
},
"speakers": {
"heading": "Luidsprekers",
"label": "Selecteer uw audio-uitvoer",
Expand Down
7 changes: 7 additions & 0 deletions src/frontend/src/stores/userChoices.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ const isVideoResolution = (value: unknown): value is VideoResolution =>
export type LocalUserChoices = Omit<LocalUserChoicesLK, 'username'> & {
processorConfig?: ProcessorConfig
noiseReductionEnabled?: boolean
audioLevelEqualizationEnabled?: boolean
audioOutputDeviceId?: string
videoPublishResolution?: VideoResolution
videoSubscribeQuality?: VideoQuality
Expand All @@ -28,6 +29,8 @@ export type LocalUserChoices = Omit<LocalUserChoicesLK, 'username'> & {
function getUserChoicesState(): LocalUserChoices {
const stored: LocalUserChoices = {
noiseReductionEnabled: false,
// Audio level equalization defaults off — opt-in before it becomes the default
audioLevelEqualizationEnabled: false,
audioOutputDeviceId: 'default', // Use 'default' to match LiveKit's standard device selection behavior
videoPublishResolution: 'h720',
videoSubscribeQuality: VideoQuality.HIGH,
Expand Down Expand Up @@ -109,6 +112,10 @@ export const saveNoiseReductionEnabled = (enabled: boolean) => {
userChoicesStore.noiseReductionEnabled = enabled
}

export const saveAudioLevelEqualizationEnabled = (enabled: boolean) => {
userChoicesStore.audioLevelEqualizationEnabled = enabled
}

export const saveProcessorConfig = (
processorConfig: ProcessorConfig | undefined
) => {
Expand Down