Machine Learning is a hot subject these days and there are a lot of new developments in the field of Artificial Intelligence. We interviewed Dr. Marc Schoenauer, Principal Senior Researcher with INRIA and a board member of MyDataModels, to talk about the challenges facing Machine Learning and to discuss the latest trends. Here is the interview:
MyDataModels: Marc, thank you for taking the time to answer our questions. To start, could you please tell our readers a few words about yourself?
Marc Schoenauer: Yes, I’m a researcher, working today with INRIA[1]. I joined INRIA in 2001 as a Senior Researcher (DR2). Before that, I spent 20 years in CNRS[2] doing approximately the same thing.
Actually, I should start from the beginning [smiling]. I graduated from ENS Paris then I got a degree in Applied Mathematics (Numerical Analysis) and gradually turned to Optimization, more precisely Stochastic Optimization, Evolutionary Algorithms and Machine Learning. In 2001 I joined INRIA as a Senior Researcher and became Principal Senior Researcher (DR1) in 2007. I’m co-head of the team at INRIA Saclay that was called TAO (Thème Apprentissage et Optimisation, Machine Learning and Optimization in English) and that I have co-created with Michèle Sebag in 2003, as a joined team between INRIA, CNRS and University of Paris-Sud. I have been President of the French Association for Artificial Intelligence (AFIA – l’association française pour l’intelligence artificielle) from 2002 to 2004, I am Chair of SIGEVO, the Special Interest Group of ACM[3] about evolutionary computing, term ending this summer after four years. And maybe something rather visible recently, I was a member of Villani Mission that wrote the report on the French Strategy for Artificial Intelligence in March 2018.
MyDataModels: Could you share with us the types of research you conducted and the findings that you made, what are your career’s big achievements?
Marc Schoenauer: I started in Applied Mathematics, I worked quite a lot in evolutionary computation. I applied these methods to solve some problems of numerical analysis. In particular I did some work in Topological Optimum Design [4] designing structures in Solid Mechanics. One of the highlights of this activity was the design of “evolutionary” chairs. An architect, Philippe Morel, came to see me and said “I like the work that you’re doing in solid mechanics and design. As an architect, I’d like to work with you”. And he added “Well, we could start with what all first-year students in architecture do – design a chair” So did we. For me, it was not a game but some fun stuff. It happened that I had a student who could start working on this and so we designed a chair. But what surprised me is that Philippe actually built some of these chairs in his garage, out of plywood. He then presented them at some architecture exhibitions. Eventually, it was spotted by Beaubourg[5], and they bought two chairs, together with the description of the process, and this now stand in their permanent collection.
Also, to continue about my career, I started to work on finite element meshes[6], you know, when you partition the space in small triangles or small tetrahedrons. One problem is to predict a priori (before running any simulation) the quality of given mesh. This is some kind of optimization problem, but it also calls on the experience of experts. And this was the time of the triumph of expert systems[7] and so I started to work with expert systems, together with Michèle Sebag. However, it soon appeared that if you want to use some expert system, you need rules, and nobody knew the rules. Hence you needed to learn the rules, and this is how I came into machine learning, somehow closing the loop with optimization. And in fact, this is also how we created a machine learning and optimization group at INRIA later.
Maybe I can share another highlight. I am none of the author, but it was done in our Tao project-team. Oliver Teytaud, member of Tao, is the main architect of the MoGo program, a Go-playing program, one of the ancestors of AlphaGo (by DeepMind). MoGo was the first computer program to beat a professional Go player at the 2009 Taiwan Open.
MyDataModels: Recently, you’ve joined the automated machine learning company MyDataModels as a board member. What made the company attractive to you?
Marc Schoenauer: Well, I have been working in evolutionary competition for some time, as you understood. I met Denis[8] first, he came with an original scientific product and a company project to evaluate it and put it on the market, which is not so frequent. It also seemed to be quite efficient on small data, which is, somehow, the counterpart of today Big Data hype. And I felt that there is also some need for this. Also, I liked the idea of using Genetic Programming (GP). This can be seen as a comeback of GP: GP has been around since the early 90s, but there have not been so many functional and commercial GP implementations. To summarize, I liked this project because it offered a way to handle small data using GP.
Interestingly, nowadays the big hype is around deep learning, but even in the context of deep learning we witness today some comeback of evolutionary algorithms. People are using evolution strategies for direct policy search in reinforcement learning; people are using genetic programming to evolve the structure of deep networks, and MyDataModels proposes another possible comeback of evolutionary algorithms, though not at all related to deep learning. But I am well aware that even if you have the best engine there is a lot more work to be done to make it a product, to market it, so I liked the fact that somebody was ready to do precisely that.
MyDataModels: You mentioned above that there is a comeback of popularity of evolutionary algorithms, is there a particular reason for this?
Marc Schoenauer: Maybe we are hitting the limits of other processes. There is a saying that “evolutionary algorithm is the second-best solution for every problem”. Meaning that EAs will be able to solve problems out of reach of more traditional approaches, until someone proposes a very specific method, and beats the evolutionary approach. So, to come to the question: My feeling is that there is indeed today a comeback of evolutionary computation. It might be a surf-on-the-wave of machine learning . And at some point in the future, probably, people will find another way to do the same thing, but it might be in 2, 3, 5, 10 years so in the meantime there is an opportunity to seize.
MyDataModels: What are the top trends in machine learning nowadays?
Marc Schoenauer: The significant part is deep learning, it is everywhere and it deserves it. There are big successes with deep learning, but it doesn’t solve everything.
Maybe the first one I would mention here is an automated machine learning that anyone can use. And this also concerns deep learning. Today, you need to be a data scientist to work with deep learning, or Machine Learning in general. Hence researchers are working on making it available to everyone. It’s called AutoML (Automated Machine Learning), or AutoDL (Automated Deep Learning).
Another trend is Small Data. Deep learning is concerned with Big Data, you need millions of examples to train a deep network, something you cannot do with a small dataset. But many activities do not have that many data: There is a lot to be done for Small Data today.
One more trend is sequential learning and decision-making, open-ended learning. You put an autonomous robot in some environment, it is on its own, so it has to learn and act, interact with the outside world and learn because everything is evolving. A lot of people are working on this, and a lot of different domains are involved.
MyDataModels: What opportunities/challenges exist for MyDataModels in your opinion?
Marc Schoenauer: Well, of course, one of the challenges is to reach millions of potential users. Because many people have data. As a scientist I see people coming to me and saying, “I have this data, what can I do with it?” The first answer is that only you, the end user, can answer the question of what you can do with your data. It’s not a data scientist who doesn’t know your job who can tell you. First, try to answer the question “What can I do with my data”. Of course, you can have a dialog with scientists or people from MyDataModels to try to come up with something that you could do with it. But only a domain expert can answer this initial question. Hence, the challenge is to educate people enough so that they can foresee what they can do with their data. They may have wrong ideas or no idea at all of what can be useful for their own company, for instance.
MyDataModels: What advice would you give to researchers who want to use their data and machine learning in their job?
Marc Schoenauer: First, as said, they need to know what would be good for them, for their job but also staying in the bounds of what is possible. We are talking about researchers, so they are getting more and more educated. They should know the quantity of their data, its quality too because data can be noisy or it may have missing values, etc. They should not choose the tool before they know their data and what they want to do with it. Also, it would be my advice to sales people at MyDataModels: do not pretend to solve everything, you have to chat with the person, and maybe you have to say “no, we cannot help you.” And it’s also good for the image of the company.
[1] French Institute for Research in Computer Science and Automation
[2] The French National Center for Scientific Research
[3] ACM’s Special Interest Groups (SIGs) represent major areas of computing, addressing the interests of technical communities that drive innovation. SIGs offer a wealth of conferences, publications and activities focused on specific computing sub-disciplines. They enable members to share expertise, discovery and best practices.
[4] Structural topology optimization is addressed through Genetic Algorithms: A set of designs is evolved following the Darwinian survival-of-fittest principle. The goal is to optimize the weight of the structure under displacement constraints.
[5] National museum of modern art
[6] Finite Element Method is the most commonly used method in numerical analysis. It was originally developed for solving problems in solid-state mechanics, but it has since found wide applications in all areas of computational physics and engineering
[7] Expert system, a term in artificial intelligence, is a computer system that imitates the decision-making ability of a human expert
[8] Denis Bastiment, co-founder and CTO of MyDataModels
