Isolation, pandemics, and physical distancing have been hard for everyone, regardless of infection. Social over Zoom is not quite the same and my weekly DS9 watch parties have lost a bit of their energy, but we move on.
Same goes for my work. Normally, I like to work and fix problems that I found around me and these days my environment is severely limited (see sidebar). They say it is a great time to write a novel, but I am not a great writer and I would rather build something. I have a lot of meetings in the morning and then heads-down work in the afternoon. I have been meaning to build a new project for some time but have not had the time to get it started. Plus, a lot of my fun projects are due to some new piece of hardware and I have not gotten any as of late. I could build classifier, which would be easy in my situation, because I have one state: this room, working, and me sitting. There is not much around me besides the sounds of my fan and my dog snoring at my feet.
What can I do about being bored and boardless?
Step in Edge Impulse!
Edge Impulse is a new startup aimed at making ML on embedded devices easier for everyone. If you have ML experience, great, if not, no worries. They make it easy. It breaks down into several discrete steps the process of training models and testing them. Here, they are called Impulses, which are really a combination of models with pre-processing options. The important and the amazing feature is that it trains automatically. Think of it as the next step towards fully automatic Machine Learning. If you don't know how to train a model, don't worry. If you do know how to train a model, you can switch to the Jupyter-like full development mode. It's all running Keras, so if you're familiar with that, then you're good to go.
Edge Impulse runs TensorflowLite Micro under the hood, so it only works for a few types of problems and on a few boards, but it is still rife with opportunity. If you want to classify vibration or sound, those models and impulses have existed since Edge Impulse launch, and as of last week, simple object classification using computer vision.
Last week I had the privilege of being on an Innovation Coffee with Arm Innovator and OpenMV co-founder Kwabena Agyeman where he announced the OpenMV + Edge Impulse Integration. This adds more tools to OpenMV and more hardware to EdgeImpulse, and the combination of both mean it is easier than ever to develop TinyML for Computer Vision applications.
Luckily I have been interacting with them for some time as they are an AI Partner, but negligent in my evangelist duties... until now.
The Edge Impulse documentation is extremely well thought out and features a few tutorials. I went with the audio classification tutorial. It makes it extremely easy to get started, Edge Impulse even has the ability to use your smartphone as a capture device. It means prototyping is fast. The only question left once I finished was what to classify to make it my own given my limited environment?
This is Porkchop. She does not do much besides be cute and snore. She sleeps most of the time, as visible in the previous photo of my desk (did not even see her, did you?) She also snores, a lot, as flat-faced dogs do. So? Time to classify those snores. Edge Impulse makes grabbing data easy and training models easier. Through the web tool, you can use a smartphone or your end-device to gather data. I chose just to stick with the phone for now. The steps are:
Testing can be done with the web tool using previous data or on device. Each of these steps is well documented, but some of the pre-processing tools could be a bit better. I had to do some reading and learning. Luckily, depending on your data type, Edge Impulse makes some good recommendations and I found these to be very good for my use-case.
Where I did run into problems was my data. As of this post, Edge Impulse does not have any data editing tools. But, Edge Impulse is a brand new tool and the team is busy rolling out new features all the time, so expect things like this to change. Feel free to make feature requests on their forums.
The issue with the data is this: normally, you want very clear lines between your categories. Given that I was naturally capturing my sounds and recording 30-60 second blocks of dog snores, those are interspersed with silence, or "background". It means that my "snore" data is roughly %50 to %70 identical to my "background" data. This is a common data problem. Sometimes you cannot force ideal conditions depending on your problem set.
The fix is easy: edit out the background noise and have the "snore" data consist only of snores. Luckily, there are a good bit of python libraries around visualizing and editing audio data, most notably librosa, and pydub. Between the two of these I were easily able to snip out all my unwanted empty noise and have beautiful, full snore samples. A notebook for that can be found here:
There were a few other methods I didn't try: K-Means to separate it out being a big one. I also manually edited the first one just to see if I could get a spike in my performance as a sanity check.
EDIT: Someone handily sent me this paper if you are interested in K-means like capabilities for audio.
Probably the best feature of Edge Impulse is the ease it deployment. You can export your built Impulse as a binary library for C++, Arduino, or Cube.AI CMSIS-PACK. You can also export it as a WebAssembly Package to run in a browser. This means that most Arm Cortex M and Cortex A based hardware are supported with a simple library import.
For even easier deployment, Edge Impulse is also working on growing its library of firmware so that you can directly deploy to supported boards. Right now those consist of the ST IoT Discovery Kit, Arduino Nano 33BLE, and the Eta Compute ECM3532 AI Sensor.
My next steps are going to be to deploy this to an Arduino Nano and build it into a collar.
What's your next step?
Get started with Edge Impulse