Building an intelligent voice controlled mirror

The project is popularly known as the “Magic Mirror” in the DIY community. Here’s my attempt.

I came across the ‘Magic Mirror’ project when searching for a mirror online for my house. To my surprise, most decent looking mirrors cost a bomb. I was in no mood of spending that much money on a mirror that did nothing apart from showing me my ugly face every morning. It could definitely do more. Initial thoughts involved vague ideas like the mirror greeting every time it detected a face, a simple mirror with a wooden frame I’d make myself… blah blah. I found myself completely dumbfounded.. anyway let’s cut the BS… I wanted it to be cool, something to show off. A quick internet search got a lot of things in place, and the Magic Mirror project was found (rather we found each other… okay too sentimental for an inanimate object there).

You will mainly need six things to make the Mirror, the last 3 are absolutely necessary.

1. an always on computing device (preferably a Raspberry Pi)
2. an LCD screen (a monitor screen is a great option)
3. a one-way mirror (or a glass and a one-way mirror film)
4. willpower (oh! things will break I assure you)
5. patience (mainly for stuff to arrive, code to work, stuff to arrive when the stuff you ordered earlier breaks and burns)
6. money (or at least the ability/access to procure money/parts)

Everything else is peripherals (like wires, cables, power supply) and does not deserve a special mention so early in the post. Knowledge of programming is an added advantage. If you wish to make the mirror pretty, woodworking skills might be handy (I honestly wanted it to be sexy af, so I got someone to do it for me). I definitely took a longer route to make the mirror. Many of the processes can be shortened. I learned a new thing at each step, so I guess it was worth it.

This is not exactly a ‘how to’ article. I trust you to figure out certain things like procurement of components, bugs, etc. as you go along. So if you were expecting that, then I’ll save you some time. Else read on. It will be fun… I promise.

Some parts of the mirror have been upcycled. There have been a number of attempts at making a voice controlled mirror, most make use of a python voice API, I also found the Google assistant and the Amazon Alexa skills API versions. I however, have made a custom web app that runs on a local server using multiple of APIs to do different tasks.

The magic mirror consists of 3 major components

1. Hardware for computation
2. Software for voice control
3. The mirror

The first two components are absolutely essential for the mirror to work functionally. The third one depends on your aesthetic inclination; if you are ok with using a black mirror, you can ignore component number 3, although I strongly suggest against it.

Hardware

Hardware includes all the physical aspects of the mirror you would need for computation. This does not include the aesthetic components of the mirror like the frame, glass, etc. Here is what you would need

A Raspberry Pi

Raspberry Pi is a mini-computer. This is where all your software goes in. I used a Raspberry Pi 3 for the project, however you are free to use any version of Pi. The advantage of Pi 3 is that it has inbuilt modules for bluetooth and wifi as well as an HDMI port. For other versions, you’ll need to install these separately.

MicroSD Card

You’ll need a MicroSD card with at least 2GB of storage for the Pi. I’d suggest getting a 4GB one.

Mini USB Cable with Adapter

The Pi is powered through a Mini USB cable with a 5V adapter. I found that it works just as fine with any mini USB phone charger.

Raspberry Pi Case (Optional)

I suggest getting a case for the Pi for the sake of safety. There are lots of cool cases available online.

Microphone

If you are building a mirror with voice recognition capabilities, this is an absolute necessity. I used a cheap webcam with a microphone. The webcam hasn’t been used in the project, however the hope is that the ‘Mirror' will someday get face recognition capabilities.

Microphone Adapter

Surprisingly enough, the Raspi 3 does not have have prebuilt support for microphones. You will need to get a microphone adapter and install drivers for the same. I will be covering all the aspects of setting up the Pi and installing drivers in the next section.

Monitor/LCD screen

The Magic mirror is essentially an LCD screen behind a one way mirror glass. You’ll need to get a working monitor (any size), preferably with HDMI support. The mirror interface works better if the glass is as close to the screen as possible, so it is better to strip off the plastic bezels and casing around the monitor. Do this very carefully. Here is a link that’ll help. I converted an old laptop screen into a monitor. Here’s a link to how you can do it.

HDMI Cable

You’ll need this to connect the monitor with the Raspberry Pi. If your monitor doesn’t have HDMI support, you’ll need an HDMI to VGA converter.

Mouse and Keyboard

You’ll need this during the setup and for fixing bugs.

Software

Follow these links step by step to get started

1. Setting up your raspberry pi 3 (sigh, it doesn’t work out of the box)
2. Setting up microphone drivers
3. Hosting a local server (only the first half is important here)

WebAPP

I made a sample web App for the ones not too enthu about writing code. You can find the link to the GitHub page here. It wouldn’t work immediately. You’ll need to add API keys for some components.

Here’s where to find API keys

1. Wolfram Alpha (Sign up and create the key. Add the key to the file name wolfram.js)
2. OpenWeather (Add key to the file name weather.js)

Then host the WebApp on the local server you created. Here’s how you can do it.

Note: Use Chromium for best results. Go to localhost/magicmirror (or wherever you have hosted the file on the server). The Wolfram API might give you a Cross origin request error, I used a plugin that allows CORS. Make sure you switch it on before using the app. Reload if nothing works. You will need a working internet connection for the mirror to show weather and respond to voice. Without internet, it works as a clock. You are free to modify the app as you wish. Don’t forget to give credit.

Conceptual diagram showing how the app works

Following are the keywords you can use to invoke the mirror. Stand in front of the mirror and say these out loud

1. “Hey yo”: This will greet you. By default it calls you sexy. So much for motivation.
2. “wiki”: follow this with a request like “wiki Albert Einstein”
3. “dodo”: follow this with any question like “dodo what is the distance between Hyderabad and Bangalore” or “dodo what is the area of a circle.” Dodo answers most questions well, he fails often though. Which why he is called that.

A video of the working AI

The User Interface

The primary function of the Magic Mirror is to be a mirror first. The embedded intelligence and information presentation capabilities are second. It was a conscious choice for the UI to be as minimal as possible and be there only if it was absolutely necessary. The mirror is entirely operated by voice, so the answers had to be temporal and short. The answers disappear after about a minute. In case of absence of an internet connection, the mirror still works as a clock and shows the current date.

The mirror needs a high contrast ratio for the sake of legibility. A black and white color palette is used since this combination has a high contrast ratio without it interfering too much with the mirror reflection. All elements are white since the color emits the highest amount of light while the background is black since it emits the lowest amount. Here’s the UI detail.

The Mirror

This step is optional but I strongly suggest you don’t skip it. For the setup above to work as a mirror, you’ll need a few things,

A One way mirror

The One way mirror goes in front of the screen, like shown in the figure above. You can find this in any hardware store. An alternative is to get a one way mirror film and stick it to a piece of glass.

Frame

This is completely up to you. I wanted the mirror to feel like it is a part of the house and not an ugly mass of wires and cables. So I got a local vendor to create a wooden frame for the set up.

Finally this is how it looks

I am so glad to have gotten this working at last. I would like to thank stackoverflow, instructables and the good folks of the internet for helping me complete this project.

I used the following apis for the Mirror App

1. Aanyang for converting Speech to text
2. Wikipedia Search
3. Wolfram short answers for intelligent queries
4. OpenWeatherMap

Peace!