English
中文
The popularity of videoconferences and their widespread use have prompted DXOMARK to start looking at which laptops provide the best end-user experience when it comes to audio-visual communication. Our initial evaluation of laptop videoconferencing capabilities helps us identify a very important challenge of the audio experience: full duplex audio rendering. Full duplex is the ability of the device to make two-way communication possible without echo nor over filtering of the voice.
In this tech article, we tested the full duplex experience of a selection of laptops representative of the market: Will the Surface Pro 8 perform better than the MacBook Pro? Are commercial laptops aligned with the needs of telecommuters? Is it necessary to buy a premium laptop to have a good two-way conversation with friends and family? We answer these questions in the following paragraphs.
The challenge of full duplex audio
In a video mediated context, the audio channels must be handled either in half-duplex or full duplex mode. The first mode is the one of walkie talkies: one person talks at a time. The latter is like a conversation over the phone, during which one party can interrupt the other party before they are finished talking.
As two-way conversation is the heart and soul of videoconferences, DXOMARK’s “full duplex audio” tests evaluate how well the laptop handles both incoming and outgoing voices, along with background and ambient noises on both ends. What do participants hear when someone interrupts a person who is speaking? Is one voice suppressed, are both voices heard, or do both voices disappear? Do you hear an echo when you speak? In group settings, can the people furthest from the laptop microphone be heard clearly? How effective is noise cancelation? Is the software able to compensate for noisy environments?
Audio defects of full duplex mode
Some of the most frequent audio defects observed in a full duplex conversation are:
- leakage
- temporal artifacts
- spatial issues
- poor tonal balance
In this article we will focus on the two most critical ones: leakage and temporal artifacts.
Double-talk leakage
Full duplex audio is difficult to implement because the playback of the incoming voice channel may be sent again into the outcoming one, creating an audio loop and sometimes Larsen effect. Leakage happens when the audio played back on the laptop ends up in the captured audio and transmitted back. It is therefore necessary to implement ambient sounds filtering and echo cancellation on both sides of the conversation to make it possible to have a telephone-like performance.
Temporal artifacts
Temporal artifacts can be summarized as any changes in volume or dynamics over time that are unwanted and/or disturbing to the end user. As mentioned above, all laptops are tuned to filter out sounds below a certain loudness threshold to allow the “main signal” to get through without the interference of extraneous noise. Moreover, noise reduction is dependent on noise filtering algorithms. This leads to the apparition of Temporal artifacts.
Temporal artifacts can lead to removing part of the sound of the voice of the person talking, a phenomenon known as gating.
Consequences of the defects
Echo cancellation
Hearing an echo of a participant is a direct consequence of the leakage defect. It can be amplified by factors outside of the device itself, such as the latency of the internet connection or the loss of packets during transmission. To stay at the device level, DXOMARK evaluates in priority the intrinsic leakage of the laptop under test, outside of the network aspect. The way manufacturers can reduce this issue is by implementing echo cancellation algorithms at the firmware level.
Gating due to ambient noise
Another big impediment to clear conversations is a phenomenon known as gating. It is a consequence of temporal artifacts, which occurs when devices are not tuned to properly differentiate between main signals and unwanted noise. Unwanted noise can be the sound of the keyboard of the laptop, some people talking in the background, or street sound coming by the window. Heavy gating can lead to simply suppress the voice of the speaker at the same time as what is perceived as noise.
The recent introduction of AI base noise reduction in the most advanced chipsets helps to improve this situation, sometimes removing gating completely; however, these algorithms are still limited to high-end solutions.
Gating due to voice priority
The double talk case, where both users speak at the same time, is difficult to manage correctly, and ends up often by one of the persons in the call taking “voice priority” over the other participant. Who has not experienced failing to make their viewpoint heard at a meeting, with many attendees, because another participant kept talking without stopping?
This phenomenon can lead to frustration, especially because we expect the same experience we are used to on a phone device. But in a conference call, the laptop manages multiple channels, sometimes over ten, while a phone only handles two channels. A very difficult algorithmic challenge for both DSP makers and laptop vendors.
Our test setup
We developed a complete videoconference testing protocol, which covers video quality, audio playback and capture, as well as full-duplex audio. For the case of full duplex, 360° sounds recordings simulate meeting environments and a duplex scenario, in which the device plays content while recording. We also perform real-life tests in various venues, using common V/C software, such as Microsoft Teams, Google Meet, or Zoom.
One of our test setups in our listening studio reproduces a very difficult use case. We use recordings of Harvard Sentences, the industry standard for speech quality measurement [1]. The scenario consists of an input channel(what the microphone of the laptop must record) and an output channel (what the speakers of the laptop are playing). The input is composed of background voices and a main male speaker, whilst the output is the voice of a woman.
In a good videoconference experience, the male speaker hears the woman properly. A perfect device in full duplex will just record the voice of the man and ignore the voice of the woman. Moreover, the recording of the male speaker should reduce the volume of the background speakers, to allow the woman to clearly hear the conversation.
The soundtrack above represents the full ambiance in front of the laptop tested.
Full duplex audio test results
In our initial round of testing, we evaluated 10 laptops from three segments [2] and a wide variety of CPUs [3].
The Acer Aspire 5 and the Asus ExpertBook show the best full duplex audio performance.
The Acer Aspire 5’s performance lands in the No.1 position in our evaluation: You do not hear yourself speaking, but gating occurs occasionally when two people speak at the same time. Our tests revealed that devices that present leakage often do not have gating, and vice versa.
The HP ProBook and the Lenovo IdeaPad 5 suffer from heavy gating and cut off the overall audio when the people on both sides of the call speak at the same time.
The Apple MacBook Pro suffers from slight duplex leakage but preserves well the vocal quality of the other person talking. This allows for easy filtering by the V/C platform afterwards. An example of bad leakage performance is the Huawei MateBook 14s, on which the woman’s voice is heavily distorted, making the voice separation difficult.
Results can also vary within the same brand: we were quite surprised to see that the IdeaPad 5 had one of the lowest duplex performances observed, while its Chromebook cousin, the IdeaPad Flex 5, figures at the fourth position. Another surprise is to see a commercial laptop, the HP ProBook 430 8G, perform poorly in duplex, given that one of the prime usages of commercial devices is to participate in online calls and meetings.
Another takeaway on duplex audio is that the price of the device is not always a good indicator of better performance. In our small but representative selection of devices, we can see that our top performer is a budget laptop, as well as the last, while premium laptops land in the middle.
Finally, remember that duplex audio is just one part of the whole audio experience, with the recording and the playback. While duplex is critical in the success of good conference calls, the ability of a laptop to have rich and powerful audio playback, as well as precise recording can sometimes make up for the flaws due to leakage or gating during duplex. Stay tuned for future DXOMARK tech articles on these subjects!
[1] “IEEE Recommended Practice for Speech Quality Measurements,” in IEEE No 297-1969 , vol., no., pp.1-24, 11 June 1969, https://doi.org/10.1109/IEEESTD.1969.7405210.
[2] To compare laptop videoconference performance in the context of their respecting audience, DXOMARK defined a segmentation system for the laptop industry, based on three dimensions: performance, design and price. Performance relates to the elements such as semiconductor components, screen technology, and storage. Design considers the originality of the curves of the device, materials and structural construction. Finally, the price refers to an evaluation of the entry point into the segment. We retained three segments for this analysis: consumer budget, mostly driven by price; commercial, which is mostly driven by design; premium, which is driven by both performance and design.
[3] Most of the Intel laptops we tested for videoconference run on 11th gen, except for the IdeaPad Flex 5 Chromebook, which uses a 10th gen, and the Asus ExpertBook, which is an older model with Intel core 8th gen. Finally, our model of Lenovo Ideapad 5 boasts an AMD Ryzen 5000, and the MacBook is the 2021 version with M1 Pro.
DXOMARK invites our readership (you) to post comments on the articles on this website. Read more about our Comment Policy.