Notebookcheck Logo

Security researchers demonstrate how to spy on screens during video calls

Could Synesthesia tell if you're on a certain site during a video call? (Source: bluejeans.com)
Could Synesthesia tell if you're on a certain site during a video call? (Source: bluejeans.com)
No-one in your video-conference session can see what you're doing with your own screen, right? Wrong, say a team of cybersecurity researchers. They have developed a new system, Synesthesia, to convert audio from your mic into the website you're looking at.

Picture the scene: you're in the middle of a regular video-conference session with your boss and colleagues, no-one is paying attention to you, and it's become tedious. So you bring up Notebookcheck News on the machine you're using to have something worth looking at while the meeting washes over you. However, the nightmare scenario of somehow being called out on your boredom-smashing behavior is always at the back of your mind.

This possibility may now become a reality, thanks to new, partially Defense Advanced Research Projects Agency- (DARPA) funded research into side-channel exploits. It suggests that cyber-spying during video calls is perfectly doable. This project has resulted in Synesthesia, a method that depends on picking up audio through the microphone of target devices for a certain portion of coil whine.

This coil whine is the high-pitched audio that pertains specifically to the power used by an LCD screen to display pixels. The researchers behind Synesthesia found that it could be isolated and converted into a visual form not dissimilar from a raster plot.

This was an impressive achievement. However, the graphical data was still complex and difficult to relate to specific screen contents using conventional means. This is where machine learning came in. The researchers reported that their neural networks were capable of processing the data with greatly improved success. Furthermore, it was able to match it to the typical display contents associated with the websites ranked 1 to 10 on alexa.com with an accuracy of nearly 97%.

Therefore, managers worried about what staff are really doing with their time may soon be able to monitor the activity - all of it - on a corporate video call. In addition, the existence of Synesthesia is yet another worrying sign of how privacy can leak out of any communications-related peripheral.

Synesthesia was designed to process mic audio picked up by a smartphone (left) for the coil whine associated with screen use (right). (Credit: Genkin, Pattani, Schuster, Tromer)
Synesthesia was designed to process mic audio picked up by a smartphone (left) for the coil whine associated with screen use (right). (Credit: Genkin, Pattani, Schuster, Tromer)

Source(s)

Read all 2 comments / answer
static version load dynamic
Loading Comments
Comment on this article
Please share our article, every link counts!
> Expert Reviews and News on Laptops, Smartphones and Tech Innovations > News > News Archive > Newsarchive 2018 08 > Security researchers demonstrate how to spy on screens during video calls
Deirdre O Donnell, 2018-08-31 (Update: 2018-08-31)