As the sun made its final attempt to heat up the Londoners, Arvid Nilsson and Jonas Karlsson from the Deep learning team at Berge flew over the North Sea to listen to discussions on this popular topic in the British capital. Here are their reflections on the two days of 'Deep learning summit', arranged by RE-WORK on September 22-23!
Data, data, data
The issues regarding lack of and trouble accessing data was a recurring topic throughout the summit. A number of different methods to overcome this issue was discussed. Raia Hadsell from Google DeepMind talked about continuous learning and especially how progressive neural networks could be helpful when data is of short supply and when one would like to further improve training time when new networks were trained.
The key idea behind these networks is the utilization of a concept called transfer learning. First you train a neural network for a specific task and then you reuse that network when you are training a new network for a new task. This is done by feeding the new input data to the old network and then feed the data back from the old network via adaptor blocks. During training the weights in the old network is frozen and only the weights in the new net and the adaptor blocks are being updated. This could be a very nice approach for training neural networks on synthetic data in a simulation and then utilize that network in a new network trained in a real world environment.
Deep learning in Health care
A lot of focus on the second day of the summit was on health care. Jeffrey de Fauw talked about how to detect diabetic retinopathy with convolutional neural networks. These kinds of networks could potentially be of great help in the future of medical diagnostics. Another topic in the field of health service was about how we can utilize AI to predict if a novel drug would be safe and efficient before even trying it out.
Ali Parsa from Babylon Health gave a very inspiring talk about personalized health service. He spoke of how we can make the health service more accessible for people in poor areas and how health service can be more efficient and of higher quality in general.
Self-driving cars and robotics
Magnus Posner from the University of Oxford talked about the future in self driving vehicles and how deep learning is applied in that field. He mentioned some very interesting concepts about how to learn a reward function for reinforcement learning by evaluating data from vehicles driven by people.
Deep tracking technology was presented by Peter Ondruska, also he from the University of Oxford. Peter presented how neural nets can be trained to very effectively track objects from raw laser data. He also showed some very impressive 3D reconstruction networks.
A number of different startups were talking about how they apply deep learning in their applications. Two examples worth mentioning are Tractable and AI Build. Tractable has built a tool to significantly lower the time it takes to label images and AI Build uses deep learning to make 3D printing adaptable during printing. They used a vision system to evaluate the height and structure of the surface on the go.
Deep learning in chatbots
The chatbot track in this years RE-WORK was interesting to follow and here are some notes of relevance:
- Consensus is that the demand and the market is ready but the technology is not quite there yet.
- Tay from Microsoft got quite a bit of bashing.
- Everybody is starting out with wit.ai (acquired by Facebook in January) or API.AI (acquired by Google late September) but most developers move on to creating their own framework due to scaling issues.
- A couple of shameless plugs but in general great presentations and really interesting applications. (Tina the T-Rex, admitHub, your.MD, etc.)
Link to the event: https://www.re-work.co/events/deep-learning-london-2016