Rob Aitken is an Arm Research Fellow. Here, he shares his thoughts and key takeaways from the Future Chips Forum, held last December in Beijing.
'Smart Chips, Smart World' was the theme of this year's forum, organized by the Innovation Center for Future Chips at Tsinghua University in Beijing, and brought together a set of mostly invited speakers on a wide variety of subjects related generally to the intersection of AI and chip design. I gave a talk on “Distributed Machine Learning for Large-scale IoT Systems”, the slides for which are available here, or at the bottom of this article.
In addition to the talks, there was a good opportunity to interact with the faculty and with local chip and Machine Learning startups, as well as with the assembled speakers. I even ran into former Arm intern Shaodi Wang.
The key part of the forum was the presentations. Unfortunately, the slides have not been published, but I have summarized some of my favorite talks below and provided links to similar work where available…
The first session was an overview of work at Tsinghua, which ranged from Neural Turing Machines to Software-Defined Chips. Shaojun Wei, the forum host and director of the Institute of Microelectronics, described humans as 'many-sensored actuators with complex unknown decision processes', which I found amusing, presumably due to some complex unknown classification scheme of mine. They are working on some interesting chips. MIT Technology Review have published a good overview here.
Next was a panel on whether emerging technologies would overtake CMOS in the era of AI. It went mostly as you’d expect, but there were a couple of moments worth noting. Kaushik Roy of Purdue University compared the power consumption of Deep Blue for chess (15kW), Watson for Jeopardy (200kW), and AlphaGo for Go (300kW) to show that matching human behavior in games does not come easy. Philip Wong of Stanford made a similar point about on-chip memory needs being greater than what SRAM alone can provide. Vivek De, an Intel fellow, objected to the idea that Moore’s Law was over, but he pointed out that successive devices from bipolar through NMOS and CMOS have all become progressively worse when judged strictly on device performance.
Rob with Vivek De (Intel), Themis Prodromakis (Southampton), and Philip Wong (Stanford), at Baijia Dayuan Restaurant, a former Qing Dynasty mansion, and subsequent to that, an elite prep school.
Some notable items from subsequent sessions: Jian Pei from Simon Fraser University talked about label noise and biased data along with ways to deal with them (holdout method, subsampling, cross-validation, etc.). Martin Wong of ECE Illinois talked about some work in cloud level EDA. Haibo He of Rhode Island talked about methods to avoid hand-crafted reward signals and use internal goal representations instead, which is an extension of the work published here: 'A three-network architecture for on-line learning and optimization based on adaptive dynamic programming'.
Danny Chen of Notre Dame talked about challenges in medical AI, especially in labelling, where experts often disagree on what something is. There are also simple optical issues that give rise to AI opportunities – our microscopes have a given field of view and most diagnoses concentrate on things that are visible in a single slide, potentially missing higher order structures. This article makes some similar points, reiterating the potential of deep-learning models to help overcome many biomedical imaging challenges.
Teijun Huang of Peking University talked about a retina-like sensing chip that is part of the Beijing Brain Project. The basic idea is that a human retina has about 2M pixels and transmits about 9Mbps of data, so extensive local processing occurs in the retina itself. Understanding what that communication consists of can help build better vision systems. Brains construct mental images, so the pulse train from the retina is a kind of coding scheme. This was very interesting work – I haven’t found any publication of the chip, but some of the image processing is available here.
Masato Motomura of Hokkaido University talked about reconfigurable deep neural networks, an extension of this recent paper.
Qinru Qiu of Syracuse talked about a neurosynaptic processor that attempts to have a more analog weighting scheme than TrueNorth. She is looking at spike burst coding, varying burst duration for example, to provide better encoding. A recent paper from her group, 'A Spike-Based Long Short-Term Memory on a Neurosynaptic Processor' can be viewed here.
There were several talks on memristor-based neural systems, including one from Themis Prodromakis at Southampton which was similar to his presentation at last year’s Arm Research Summit. The key takeaway was that you can get very precise resistance-based weights by writing with small charge pulses and monitoring as you write. Writing is slow, but does not happen often. Themis showed results with 7 bits of precision, good enough for a Nature paper: 'Multibit memory operation of metal-oxide bi-layer memristors'.
Overall, the forum was well worth attending. In addition to the quality of the talks, the interaction with the other attendees and speakers was very valuable. The various speakers also had some great discussions, especially during breakfast at our hotel, where the jetlag-suffering early risers had some great discussions about emerging technologies.
Will smarter chips lead to a smarter world? No guarantees, but with high quality research going on there’s always hope.