Camera captures voices without a microphone

Yasuhiro Oikawa of Waseda University in Tokyo pointed a high-speed camera at the throat of a volunteer with one task in mind: To capture his/her voice without the use of a microphone.

Yes, you read that correctly. Oikawa and his team announced at the International Congress on Acoustics on June 3 that they used cameras to take thousands of images per second and record the motions of a person’s neck and voice box as they spoke. A computer program then turned the recorded vibrations into sound waves.

Why did they do this, you ask? Some lip-reading software programs are sophisticated enough to recognize different languages, but the end result doesn’t usually involve much more than a transcript, according to a ScienceNews article. In addition, microphones often record too much background noise, so Oikawa and his colleagues, looking for a new method of capturing vocal tones, came up with this idea.

The article explains that the researchers pointed the camera at the throats of two volunteers and had them say the Japanese word tawara, which means straw bale or bag. The team recorded them at 10,000 fps, and at the same time, recorded the volunteers’ words with a standard microphone and a vibrometer for comparison. The vibrations recorded by the camera vibrations can’t be recorded by a camera – I think you mean “interpreted by the camera data) were similar to the ones from the microphone and vibrometer, Oikawa said in the article.

After running the images though a computer program, the team reconstructed the volunteers’ voices well enough to hear and understand them saying tawara. Mechanical engineer Weikang Jiang of Shanghai Jiao Tong University in China noted Oikawa did not play audio of the reconstructed voices, but instead showed the comparison photos of the sound waves and vibrations.

Like Weikang, I am interested to hear what the audio sounds like.

Personalized advertising with facial detection

“Cara” is new facial detection software from IMRSV that uses a standard webcam to scan faces up to 25 feet away and determines age and gender. It’s currently being used on a wall of shoes in the back of a Reebok store in Fifth Avenue in New York, where it is helping the store to see which customers are spending more time at the shoe wall, quickly walking away, or actually buying something.

If this experiment goes well, Reebok could install an advertising display that would intelligently react to different customers. For instance, if I were to walk into a store and pick up a pair of size 10 running shoes, a video might pop up on the screen to tell me about these shoes.

No, really.

According to IMRSV, Cara collects data with 93% detection accuracy. Its demographics include gender (92% accuracy), and age (Child, young adult, adult, senior, with 80% accuracy).  It detects at a distance of up to 25 feet away and can scan multiple people at the same time. In addition to customized marketing, Cara could be used to watch audiences during live performances and monitor whether drivers are looking at the road, says IMRSV.

While this is all quite fascinating, I can’t help but think of a scene in Minority Report, where Tom Cruise is walking down a hallway rather quickly and the digital billboards are bombarding him with personalized ads. (Check it out here.) While the technology isn’t nearly as intrusive—it’s certainly not scanning your retina and immediately placing exactly who you are and where you’re from—it is eerily reminiscent of the futuristic adverts portrayed in the film.

Tele-rehabilitation booming with Kinect

At a panel discussion at the American Telemedicine Association trade show,  Dr. Kouroush Parsapour said that physical therapy in the United States is approaching crisis, so much so that by 2030, the number of states with sub-standard physical therapy will increase from 12 to 48.  With this in mind he created 5plus Therapy, a startup that works on building digital health physical therapy tools.

At 5plus Therapy, Parsapour uses Microsoft’s Kinect to measure a patient’s movement, a task that he had previously performed with a goniometer. Parsapour is not alone. A number of tele-rehabilitation startup companies nationwide are using the Kinect. 

Reflexion Health, has started clinical trials to validate the technology. Reflexion offers a rehab measurement tool that uses Kinect to instruct the patient on exercises and measure whether they are performing their exercises correctly. 

MobiHealthNews has a list of nine companies that are using digital rehabilitation solutions, all of which use or plan to use the Microsoft Kinect. 

Microsoft Kinect can be used for rehab, weightlifting, medical record review, death mask creation, tracking workers and identifying car defects.   

Last week I wrote about how video games may be able to help improve 3D vision in adults with lazy eye. In that blog I mentioned how I was never a fan of video games, but with all of the good they are capable of, should I give them a second chance?

Cloudless atlas of earth

Using two years’ worth of images taken by Nasa satellites, a mapping site called MapBox has created a brilliant cloudless atlas of Earth.

The process begins with Nasa’s Moderate-resolution Imaging Spectroradiometer (MODIS) technology, which can image the entire Earth every one to two days. The system is attached to the Terra and Aqua satellites that were launched into orbit in 1999 and 2002, respectively. The satellites and the MODIS system images collected data using the visual field wavelength, and once MapBox received the data it wanted from Nasa, it began processing the images.

Prior to the launch, MapBox cartographer Charlie Lloyd told the Daily Mail that the 339,000, 16-megapixel+ satellite images totaled more than 5,687,476,224,000 pixels. Lloyd and fellow cartographers at MapBox then began the process of identifying images that had a clear view of the ground. By processing the images, the team was able to remove the clouds.

This gent and his team did this for every pixel in the world! This enables folks to see images of Earth that have never been seen before, including things like land-use patterns, deforestation, cities, and so on. The images created by MapBox essentially provide an idea of what astronauts on board the International Space Station see on a clear day.

While all of the images are undoubtedly impressive, a select few are truly and utterly remarkable. Take this one, for example, which shows a clear image of the UK. In this image, you can see London, The Brecon Beacons in Wales, and the highlands in Scotland.

Rather cool stuff, no doubt about it. If you’re interested in reading more, click here.

Restore 3D vision with video games

I’ve never been particularly interested in playing video games. Considering the fact that they can sometimes lead to addiction and/or violence, it actually makes me question whether or not they are best left alone, at least for young children. On the flip side, though, are the potential positives that games can bring to the table.

As it turns out, playing the right video games may actually help you improve brain function, lose weight, and…restore 3D vision for people with a lazy eye? If you had to read that last part twice, you aren’t alone. A study performed at McGill University has found that playing video games with both eyes can dramatically improve vision in adults with lazy eye, which is a condition that was thought to be all but untreatable in adults, according to a CBCNews article.

Lazy eye, also known as amblyopia, is an eye disorder characterized by impaired vision in an eye that otherwise appears normal. This is a condition that is estimated to affect 1% to 5% of the global population. Those with the condition have limited depth perception and hence cannot judge distances as well as people with normal vision.

With the new treatment developed by a team led by Robert Hess, director of the opthalmology research department at McGill and the Research Institute of the McGill University Health Centre, vision in the weaker eye of someone with lazy eye drastically improves, and rather quickly. Here’s how they did it:

They chose the game of Tetris, a game that can only be played effectively using both eyes. By splitting the image between eyepieces of head-mounted goggles, one eye sees the falling pieces and the other eye sees pieces already fitted at the bottom of the screen. After playing Tetris for an hour a day for two weeks (that’s a lot of Tetris!), nine adults with lazy eye showed vast improvement in the vision of the weaker eye and in their 3D depth perception.

Rather cool stuff, if you ask me, but I can’t say for certain whether or not I’ll be playing Tetris any time soon.

The future is here

Science and engineering aside, what is the first thing that you think of when the idea of 3D holographic images comes to mind? Star Wars? The Jetsons? Red Dwarf? For decades, the idea of 3D holography has been referenced in pop culture. So when William Hanna and Joseph Barbera portrayed the Jetsons’ using holographic televisions and telephones in 2062, just how grounded in reality were these depictions?

As it turns out, the Hanna-Barbera duo was onto something.

3D holograms are already being used to create maps that enable soldiers and commanders to navigate the terrain in which they are operating without 3D glasses or goggles. The same technology could be making its way into people’s homes and offices sooner than Hanna and Barbera might have thought.

A job listing from Microsoft suggests that the company is working on telepresence technology that would depict a virtual hologram of the person on the other end of a conversation. In other words, Microsoft is reportedly bringing 3D holograms to Skype, says Laptop Mag.

We’ve seen similar technology developed already, as researchers at Queen’s University created a human-sized 3D videoconferencing system that allows people in different locations to communicate as if they were face to face. But with the Skype hologram technology, no pods and no sensors would be involved.

Needless to say, this could revolutionize the way that offsite colleagues and business partners interact with one another. On one hand, it would be beneficial for those who are unable to meet in person for one reason or another. Meeting and chatting face-to-face and in person is something that cannot be replaced. But on the other hand, will the technology begin to erode the need for a common, shared workplace? Conjecture, no doubt, but it is interesting to think about.

In considering some of the advances involving 3D technology we’ve seen of late, what’s next? Here's what I'm thinking.

Respect the past, create the new

Roughly translated from Japanese to English, the phrase onkochishin means “Respect the past, create the new.” For this particular blog topic that advice adheres well, as scientists have produced an audio file from a 128-year-old relic.

Alexander Graham Bell, the man who is credited with inventing the first practical telephone, does not have a voice. This is not to say the man was a mute--he was not--but given that he passed away nearly 91 years ago, nobody has actually heard his voice for nearly a century?

Until now.

The Smithsonian National Museum of American History, working in tandem with the Library of Congress and Lawrence Berkeley National Laboratory, has identified a recording of Bell’s voice for the first time. It all began when a transcript that was signed and dated by Bell on April 15, 1885, was matched with a wax-on-binder-board disc that carries his initials with the same date. The Smithsonian sent the disc to go through the noninvasive optical sound recovery process  on equipment developed by the Berkeley Lab to it to be audibly matched to the transcript, and to produce an audio file.

How? Well that, of course, is the interesting part.

In this process, 3D optical metrology and surface profiling methods create a 3D digital map. The map is then processed to remove evidence of wear or damage, and software calculates the motion of a stylus moving through the disc’s grooves, reproducing the audio content into a digital file. An in-depth look at how this technology was developed and it utilized (how it is used) can be found here.

The group that produced the recording was also responsible for retrieving 10 seconds of the French folk song “Au Clair de la Lune,” from an 1860 recording of sound waves made as squiggles on a piece of paper.

So while it may not be the high-quality audio that folks today are used to today, it is only fitting that the man--who may or may not have invented the telephone--now has a voice.