Back

Article

Interview with a Senior Researcher in Machine Learning, Dr. Marc Schoenauer

Machine Learning is a hot subject these days and there are a lot of new developments in the field of Artificial Intelligence. We interviewed Dr. Marc Schoenauer, Principal Senior Researcher with INRIA and a board member of MyDataModels, to talk about the challenges facing Machine Learning and discuss the latest trends. Here is the interview:

MyDataModels: Marc, thank you for taking the time to answer our questions. To start, could you please tell our readers a few words about yourself?

Marc Schoenauer:

Yes, I’m a researcher, working today with INRIA[1]. I joined INRIA in 2001 as a Senior Researcher (DR2). Before that, I spent 20 years in CNRS[2] doing approximately the same thing.

Actually, I should start from the beginning [smiling]. I graduated from ENS Paris then I got a degree in Applied Mathematics (Numerical Analysis) and gradually turned to Optimization, more precisely Stochastic Optimization, Evolutionary Algorithms and Machine Learning. In 2001 I joined INRIA as a Senior Researcher and became Principal Senior Researcher (DR1) in 2007. I’m co-head of the team at INRIA Saclay that was called TAO (Thème Apprentissage et Optimisation, Machine Learning and Optimization in English) and that I have co-created with Michèle Sebag in 2003, as  a joined team between INRIA, CNRS and University of Paris-Sud. I have been President of the French Association for Artificial Intelligence (AFIA – l’association française pour l’intelligence artificielle) from 2002 to 2004, I am Chair of SIGEVO, the Special Interest Group of ACM[3] about evolutionary computing, term ending this summer after four years. And maybe something rather visible recently, I was a member of Villani Mission that wrote the report on the French Strategy for Artificial Intelligence in March 2018.

MyDataModels: Could you share with us the types of research you conducted and the findings that you made, what are your career’s big achievements?

I started in Applied Mathematics, I worked quite a lot in evolutionary computation. I applied these methods to solve the problems of numerical analysis. In particular I did some work on Topological Optimum Design [4] building structures in solid mechanics. One of the highlights of this activity was the design of evolutionary chairs. An architect, Philippe Morel, came to see me and said “I like the work that you’re doing in solid mechanics and design. As an architect, I’d like to work with you”. And he added “Well, we could start with what all first-year students in architecture do – to design a chair” So we did. For me, it was not a game but some fun stuff. It happened that I had a student who could start working on this and so we designed a chair. But what surprised me is that this architect actually built this chair in his garage, a wooden chair, and he presented it at some exhibitions. Eventually, it was spotted by Beaubourg[5], and they bought two chairs, together with the description of the process, and this suits in their permanent collection now.

Also, to continue about my career, I started to work on finite element meshes[6], you know, when you partition the space in small triangles or small tetrahedrons. One problem is to predict a priori (before running any simulation) the quality of given mesh. This is some kind of optimization problem, but it also calls on the experience of experts. And this was the time of the triumph of expert systems[7] and so I started to work with expert systems. However, it soon appeared that if you want to use some expert system, you need rules, and nobody knew the rules. So you had to learn the rules, and this is how I came into machine learning, which was somehow closing the loop with optimization. And in fact, this is also how we created a machine learning and optimization group at INRIA later.

Maybe I can share another highlight. I’m not the author but it was in our Tao project-team. Oliver Teytaud was in our team and he is the main architect of the  MoGo program, a Go-playing program, one of the ancestors of AlphaGo by DeepMind. MoGo was the first computer program to beat a professional Go player at the 2009 Taiwan Open.

MyDataModels: Recently, you’ve joined the automated machine learning company MyDataModels as a board member. What made the company attractive to you?

Marc Schoenauer: Well, I have been working in evolutionary competition for some time as you understood. I met Denis[8] first, he came with an original scientific product and a company project to evaluate it and put get it on the market, which is not so frequent. It also seemed to be quite performant on small data, which is, somehow, the counterpart of big data now. And I felt that there is also some need for this. Also, I liked the idea of using Genetic Programming, as it can be seen as a comeback of GP in some sense. GP has been around since 1992 but there have not been so many functional and commercial implementations of it. I liked this project because it offered a way to handle small data using genetic programming.

Interstingly, nowadays the big hype is on deep learning, and the like, but even in the context of deep learning there is some kind of comeback of evolutionary algorithms. People are using evolution strategies for direct policy search in reinforcement learning; people are using genetic programming to evolve the structure of deep networks, and MyDataModels proposes another possible comeback of evolutionary algorithms, though different from deep learning. But I am well aware that even if you have the best engine there is a lot more work to be done to make it a product, to market it, so I liked the fact that somebody was ready to do that.

MyDataModels: You mentioned above that there is a comeback of popularity of evolutionary algorithms, is there a particular reason for this?

Marc Schoenauer: Maybe we are hitting the limits of other processes. There is a saying that “evolutionary algorithm is the second-best solution for every problem” Meaning that if you dig into some narrow problem, an evolutionary algorithm cannot solve it as well as a specialized method, but if you are trying to come up with a specific solution you will be better off using evolutionary algorithms. So, to come to the question. My feeling is that there is a comeback of evolutionary computation, it can be a surf-on-the-wave of machine learning and AI now. Later, probably, people will find another way to do the same thing, but it might be in 2, 3, 5, 10 years so in the meantime there is an opportunity to seize.

MyDataModels: What are the top trends in machine learning nowadays?

Marc Schoenauer: The significant part is deep learning, it is everywhere and it deserves it. There are big successes with deep learning, but it doesn’t solve everything.

Maybe the first one I would mention here is an automated machine learning that anyone can use. And this is also true for deep learning. Today, you need to be a data scientist to work with deep learning, so researchers are working on making it available to everyone. It’s called AutoML (Automated Machine Learning), or AutoDL (Automated Deep Learning).

Another trend is Small Data. Deep learning is concerned with Big Data, you need millions of examples to train a deep network, something you cannot do with a small dataset. There is a lot to be done for Small Data yet.

One more trend is sequential learning and decision-making, open-ended learning. You put a robot in some environment, and it is on its own, so it has to learn and act, interact with the outside world and learn because everything is evolving. A lot of people are working on this, and a lot of different domains are involved.

MyDataModels: What opportunities/challenges exist for MyDataModels in your opinion?

Marc Schoenauer: Well, of course, one of the challenges is to reach millions of potential users. Because many people have data. As a scientist I see people coming to me and saying, “I have this data, what can I do with it?” The first answer is that only you, the end user, can answer the question of what you can do with your data. It’s not a data scientist who doesn’t know your job who can tell you. First, try to answer the question “What can I do with my data”.  Of course, you can have a dialog with scientists or people from MyDataModels to try to come up with something that you could do with it. But only a domain expert can answer this initial question. Hence, the challenge is to educate people enough so that they can foresee what they can do with their data. They may have wrong ideas or no idea at all of what can be useful for their own company, for instance.

MyDataModels: What advice would you give to researchers who want to use their data and machine learning in their job?

Marc Schoenauer: First, the same thing, they need to know what would be good for them, for their job but also staying in the bounds of what is possible. We are talking about researchers, so they are getting more and more educated. They should know the quantity of their data, its quality too because data can be noisy or it may have missing values, etc. They should not choose the tool before they know their data and what they want to do with it. Also, it would be my advice to sales people at MyDataModels: do not pretend to solve everything, you have to chat with the person, and maybe you have to say “no, we cannot help you.” And it’s also good for the image of the company.

[1] French Institute for Research in Computer Science and Automation

[2]  The French National Center for Scientific Research

[3]  ACM’s Special Interest Groups (SIGs) represent major areas of computing, addressing the interests of technical communities that drive innovation. SIGs offer a wealth of conferences, publications and activities focused on specific computing sub-disciplines. They enable members to share expertise, discovery and best practices.

[4] Structural topology optimization is addressed through Genetic Algorithms: A set of designs is evolved following the Darwinian survival-of-fittest principle. The goal is to optimize the weight of the structure under displacement constraints.

[5] National museum of modern art

[6] Finite Element Mathod is the most commonly used method in numerical analysis. It was originally developed for solving problems in solid-state mechanics, but it has since found wide application in all areas of computational physics and engineering

[7] Expert system, a term in artificial intelligence, is a computer system that imitates the decision-making ability of a human expert

[8] Denis Bastiment, co-founder and CTO of MyDataModels

Start using MyDataModels now, it’s free.