George Wort has been working with the Micro:bit Educational Foundation as part of his graduate rotation in the Arm School Program (ASP), from March until September of 2019. This has been an exciting opportunity for ASP to collaborate with the foundation. It also means George has worked on something that can really have a positive impact on the STEM education of future generations. I would like to thank George for his hard work and enthusiasm during this rotation.
I interviewed George to provide a reflection on his experience with a deep dive into the technology he has been working on.
"From March until September of 2019, I have been working with the Micro:bit Educational Foundation as part of my graduate rotation in the Arm School Program. This has been an exciting opportunity to work on something that can really have a positive impact on the STEM education of future generations.
For those of you who do not know, the BBC micro:bit is a “tiny programmable computer, which is designed to make learning and teaching easy and fun!”, as quoted from micro:bit. It was created in 2015 by the Micro:bit Educational Foundation, a not-for-profit organization, co-founded by Arm and the BBC, among others. It aims to give schools in the UK a way to provide a more fun and tactile platform to introduce and teach all students about programming and technology. Since the creation of the organization, their mission has expanded to a global scale and children worldwide have enjoyed and learned about technology using the micro:bit. As a device, it uses an Arm Cortex-M0 processor and provides a 5x5 LED matrix, two programmable buttons, 25-pin edge connectors, an accelerometer, magnetometer, and a radio for communicating between devices.
My main goal in the rotation was to create an in-browser simulation of the micro:bit that could run MicroPython scripts, and so be used in the MicroPython online editor. MicroPython is an implementation of Python 3 that targets microcontrollers and other such embedded devices. Alongside Microsoft’s MakeCode platform, which lets students use a block-based programming language, MicroPython is the main language that is used to target the micro:bit when teaching.
The aim of the simulator is to make it easier for users new to the language and perhaps programming as a whole. It helps them to understand and more quickly pick up the syntax and method of writing MicroPython. Previously, any error in the script would not be picked up until the student flashed their micro:bit and ran into the erroneous code. At which point, a scrolling error message would be displayed, taking a long time to read and longer to fix. Providing a simulator allows the student to quickly and often run their script as they write, making development and learning much quicker. It also allows students to explore programming and perhaps develop their programs more extensively at home where they might not have access to the device."
"The initial work involved much investigation as to the best approach, should the simulator be written from scratch. This is implemented on top of an existing project, or, perhaps another version of Python was already running in the browser and could be modified. As always, using an existing project could save much time and effort, though may restrict the project in certain ways.
Prior to five years ago, there were two options regarding implementing the simulator from scratch:
These are no longer the only options, following Emscripten’s release, it is now possible to compile C/C++ source code straight to JavaScript and WebAssembly. This allows existing C/C++ projects to be compiled to the web and run in the browser. For those who don’t know, WebAssembly is a two year old Emscripten targeted virtual ISA that can be run in-browser. Due to many reasons, WebAssembly can generally perform better than JavaScript. Using Emscripten in some form would still leave the peripherals, such as the display and sensors to be implemented in JavaScript, but none of the software stack would have to be reimplemented."
"MicroPython runs on top of two main pieces of software, the micro:bit DAL (Device Abstraction Layer), and Mbed OS. Fortunately for me, Emscripten had already been used by Arm’s very own ISG to run Mbed OS in the browser, thus requiring me to compile only MicroPython and the DAL in Emscripten. The existing Mbed simulator used the fact that Mbed OS was designed to target multiple hardware platforms using its HAL (Hardware Abstraction Layer), to target a new hardware platform, one written in JavaScript. This abstraction layer created a clean cut between existing software and simulation that allowed the hardware level, including any additional peripherals, to be simulated in JavaScript while retaining the inner workings and resources of the original software. Due to MicroPython interacting directly with hardware, at many points avoiding Mbed OS, this cut would not remain so clean.
With regards to existing in-browser Python interpreters, Skulpt and Brython are both written in JavaScript, and Pyodide uses Emscripten to compile the CPython interpreter to the browser. I decided to stick with Emscripten as these Python implementations would require a lot of reimplementation and internal modification to match a lot of MicroPython’s behaviour."
"My first big moment of satisfaction was compiling just enough of the DAL to allow the display driver to run and scroll “Hello World" across a JavaScript implemented display. At one point in my drive to produce an MVP (Minimum Viable Product) I wrote a script that produced a dependency graph between the different sections of MicroPython, the DAL, and Mbed OS that helped me to prune the code that wasn’t needed.
The biggest issues came from the requirement for the code to run asynchronously. The main interpreter was required to yield to the JavaScript simulating the hardware, the ticker handler, and the browser. Emscripten has two options for producing asynchronous code, both of which use the emscripten_sleep call to allow the current "thread" to yield to other “threads”, including the browser. The first is the compilation option Asyncify that statically analyses the code and splits it up using timeouts at the points where emscripten_sleep is or could be called. The second compiles all of the code to a custom byte-code that is then interpreted by Emscripten's interpreter, Emterpreter, at run-time. Unfortunately, neither are sufficiently able to handle setjmp and longjmp, a C/C++ feature used by MicroPython to handle exceptions. Dealing with this constraint required a large existing WIP pull request, that explicitly handled exceptions, to be applied to MicroPython. This was later wholly rewritten in order to make it more maintainable. The second largest change required was due to the MicroPython garbage collector’s inability to track certain pointers that were being held in JavaScript, this was fixed by applying another WIP pull request that explicitly tracked pointers using a stack.
The hardware ticker is implemented using set Timeout in JavaScript. This produced a good number of subtle issues due to JavaScript being single threaded and unable to pre-empt a thread as the ticker handler would not be run until the other events had ran to completion or yielded using emscripten_sleep. This led to the ticker firing at a variable frequency whatever was done. To work around this, some of MicroPython's modules were altered to take into account the time that had passed since the last tick.
After setting up a testing framework that allowed the simulator to automatically run through a series of test scripts, I discovered a considerable number of issues where a runtime exception would be thrown. It turned out that these were found because I was running the most recent set of MicroPython tests against the micro:bit port of MicroPython, which was an older version and thus contained a number of bugs that had been fixed in upstream. In order to retain simulation fidelity, these crashes would need to still occur, but in a more contained manner. To achieve this, I found the fixes required. Instead of applying them, called a custom function at the place where they occurred, that would inform the user why the crash had occurred before resetting the simulator. A similar thing has been done for calls to assert, which have been replaced with a custom mp_assert which informs the user of an internal error before resetting the simulator, rather than throwing an exception.
After many bumps and obstacles that seemed at the time like walls, the simulator now runs with the display, buttons, pins, accelerometer, magnetometer and radio. It has been integrated into the online python editor but not yet released as the UI still needs to be refined. Apart from that there is still plenty of work that can be done. There are still MicroPython modules such as music and speech that could be pulled in, as well as more features on the JavaScript side. A possibility that I would like to see is the simulation of wires connecting pins within a single instance or between multiple simulator instances.
Another possibility that could make the project more maintainable is the appearance of a recent new option in Emscripten called Bysyncify, written as a replacement to Asyncify. This may not have the same issues as Asyncify and the Emterpreter, thus allowing the two larger patches to be removed. Once the simulator is released to beta I am sure more work will become available as issues are found. All information regarding the simulator can be found here."
"Early on in the prototyping process, I took some brief benchmark results, I continued recording these results throughout the development of the project. Taking around twenty five micro-benchmarks, I took a number of rough readings to see if the existing prototyped simulator could keep up with the real thing, as well as the other simulation options. The implementation ran around 10x faster than the real thing, relieving any worries about performance. Unsurprisingly, running CPython natively was the fastest, followed by Pyodide, which was sped up significantly by using WebAssembly. It was also confirmed that the Unicorn implementation was indeed unusably slow."
"One of the most fun results of the project is being able to replicate the teleporting duck script. The teleporting duck script is a MicroPython script that uses the radio to pass a single duck between multiple micro:bits. Populating a single web page with multiple instances of the simulator, each contained within a separate iframe, such as I have made possible in the unreleased python editor, allows this behaviour to be easily replicated in a satisfying manner. Hopefully fun little things like this can show students what is possible with programming and encourage them to continue learning and exploring the dark art."
"This rotation provided me with an opportunity for a lot of independent learning and investigation into things I had previously not explored at all, This spanned from having to understand and reimplement the low-level hardware interactions in C/C++ to implementing the simulation of the peripherals in JavaScript. The Arm School Program and the Micro:bit Educational Foundation are always open to graduates wishing to do a rotation and help in their goal to improve STEM education for all. Work is available across the spectrum from low-level C/C++ tasks to JavaScript interface work, so if you’re interested, please apply."
I’d like to personally thank George for all his hard work, it’s been a pleasure working with him. We now have several more graduates doing wonderful things with the Micro:bit Educational Foundation. I would encourage anyone with an interest in this area to get in touch to see how you can contribute to this endeavour with profound impact on Computing education globally.
If you want to learn more about what the Arm School Program does with micro:bits in schools then please check out my Do Your: Bit Challenge blog.
Sign up to newsletter
For anyone interested in the source code, the entry point repo for the simulator is https://github.com/microbit-foundation/microbit-simulator, with most content for micropython in https://github.com/microbit-foundation/micropython-simulator plus a few more dependencies.