What are the AI trends that Ponicode is following in 2021 and beyond?
Hamza Sayah, Data Scientist at Ponicode, shares a quick overview of where we are at when it comes to AI innovation. He will introduce us to the key innovations driving Ponicode product development as well as some insight on emerging trends we should be keeping an eye on in order to stay on top of our industry.
Major developments in the field of AI in recent years
Here at Ponicode, we carry out intense monitoring and research work in the area of Natural Language Processing (NLP). Honestly, NLP has been around for a while. The growth of NLP began with computational linguistics, which is a set of disciplines based on statistical modelling of ordinary language approached from a computational perspective — a subject that is probably as old as computers. For a long time it was ignored by researchers, because the general consensus was that this field would never evolve significantly. However, in recent years, deep learning and new algorithms for embedding words have demonstrated an unexpected potential. Significant projects have been launched around the world in the past few years around AI and especially NLP. One of the best known is Open AI, of which Elon Musk is a member. Open AI is an association which has set itself the goal of producing "real" AI, i.e. significantly functional and independent intelligence. The most recent NLP results produced by Open AI now make it possible to imagine high value-added applications in all industries and many aspects of our daily lives.
The subject of reinforcement learning has evolved significantly as well. Open AI has produced significant advances in the field of reinforcement learning, even those that surpass humans in certain games, like chess, for example. More recently, Facebook has created an algorithm capable of beating several professional poker players. The very recent advances in the power of graphic processing units (GPUs) and tensor processing units (TPUs) have made a new wave of innovation possible. Reinforcement learning has been pushed to new heights. On top of that, neural networks are being applied to reinforcement learning to create new solutions. These new developments are leading to impressive results in terms of AI that surpasses humans on increasingly complex tasks.
NLP has developed significantly in research and in market viability thanks to neural networks. The possible applications of neural networks on NLP seem limitless since networks exist in many aspects of our daily lives: in nature, in society and in the economy. For example, neural networks can be applied to social networks. Link the members of a social network together to capture the information that connects people between themselves. The neural network will embed the network and abstract the meaning using graphs. This can also apply to Language Processing (LP) because text can be seen as a graph with words that are connected in an orderly and logical way. Capturing the relationships between graph vertices is something we see in academia, but not yet on the market, as it requires too much computer and time resources for only minimal outputs… at least for now that is.
We have moved from the linear and constant evolution of artificial intelligence to a real acceleration of discoveries in this field lately. For a while, the meaning of words felt out of reach for artificial intelligence but the recent evolution in this field has sparked a new range of possibilities in terms of machine learning. Robots are now capable of representing human language more and more meaningfully. We are getting closer to AIs that can say things that make sense; whereas meaning in language was once a characteristic unique to human beings that we didn't think was possible to share with a machine.
Ponicode & AI; what we’re interested in and the challenges we face
At Ponicode we are working to apply AI to code. We know that code is a language and we know of techniques for AI applied to language (NLP) but we only know of some discussion on how to apply it to code. The main challenge we try to tackle and the core of our research into NLP on code is focused on the subtle balance between what we know of human-to-human communication and what we know of human-to-computer communication.
A human can understand a grammatically incorrect sentence because the brain is superior to the computer in terms of understanding natural language. A computer will only be able to understand very strict language, it does not process when a full stop or comma is missing. The code can communicate very clearly on the channel developer <-> developer, but not on the channel developer <-> computer. There is a natural semantic aspect that exists so that developers can communicate with each other and a rigorous grammatical aspect so that developers can communicate with the computer. </-></->
At Ponicode, we work to apply artificial intelligence innovations to language in order to improve the interaction between the developer and the machine. This is an additional complexity to our technical challenges.
What to look out for moving forward
On a general level, I am convinced that over the next five years the field of AI will bring fundamental innovations. In terms of performance, I would not go so far as to say that there will be an AI that solves the Turing Test but at least something close to it. GPT3, for example, has huge flaws, but you can see the exponential growth compared to GPT-2. We now have AIs which understand and own the semantics of the text. It can react to the text, modify it, predict it. The applications of GPT3 will change the world. In the future we will have neural networks which can master far more complicated semantics; a sense of words, sentences, grammar and even arithmetic. A machine will be able to write grammatically correct sentences with control over the semantic as if it were a human.
Another aspect which I believe will take a large place in the near future of Artificial Intelligence is the ethics of AI. Personally, I understand and share society's concern about the dangers of artificial intelligence. The dangers of algorithms in our lives are very real. These are dangers related to the damaging power of bias, of suggestion, of influence, and of the algorithm that burrows its way into the brain. When we use any social media, instead of spending just a few minutes on the platform as we originally intend, we end up spending hours there, thanks to the recommendation system convincing us to stay. But the content that is suggested presents a risk: that of polarising society. We cannot begin to imagine to what extent algorithms can do this work of polarisation, of influencing many aspects of society. Artificial intelligence algorithms have objective functions and they try to optimise them. We have a duty to question how these functions can lead to ethical pitfalls or ethically unacceptable actions. Beyond legislation there are mathematicians and researchers who are trying to figure out how to mathematically embed ethics into algorithms. Those are wonderful challenges that will lead to many evolutions in AI.
On a final note, I believe that in the field of reinforcement learning we are going to experience a revolution in recommendation systems. Today we use neural networks for what we call deep reinforcement learning which we are seeing more and more. This is a very powerful process because we are not telling the machine to estimate something, we are telling it to maximise a reward. Now, we are starting to see reinforcement learning being used in recommendation systems. What this means is that the algorithm makes suggestions according to your profile and how you interact with something, then it is either rewarded or penalised depending on your interaction and so the machine begins to learn. This is something that I am beginning to see a lot in research and I think this combination of reinforcement learning and recommendation systems is something we will see being used more and more and in many different variations. I imagine we have not yet fully wrapped our head around the endless ways this can be used to enhance our daily lives.
Thanks to Hamza for his wonderful insights and the wealth of information he provides us with on a daily basis!
Keep learning about AI and NLP with Ponicode’s introduction to machine learning on GitHub Universe replay.
Or, for our more advanced audience, check out this enlightening introduction to Transformers.