GPT-3: the gigantic artificial intelligence
GPT-3 has become famous overnight through viral posts on social networks. Between hype and anxiety, what exactly is GPT-3? Why this AI surprised so many devs in particular? Is this game changer?
For science
Today, we will discuss artificial intelligence. And I know that this subject tends to piss off a lot of people very quickly. Especially developers. Yet, for now, AI is just a tool like any other.
I’m not going to explain what AI is or how a model works. We will only talk about what unleashes passions, generator of hype or anxiety attack depending on the point of view, GPT-3!
The company behind this technology is OpenAI. OpenAI is famous for three main reasons.
The first is that one of the founders is Elon Musk. The person who is the most afraid of artificial intelligence in the world. The other founder is Sam Altman who we will talk about later.
The second is the reason why this company exists. The goal of OpenAI is to develop artificial intelligence in a safe and sustainable way for human beings. Musk is convinced that uncontrolled AI will cause the end of humanity, so OpenIA promote a safe way to control it.
The third is that OpenAI is frequently in the news with these impressive repeated successes in its field. Notably the former GPT and GPT2 models. The last one being our hot topic of the day.
What’s GPT-3
Generative Pretrained Transformer 3 (GPT-3) is a language model that uses deep learning (neural network) to produce text. For the uninitiated, I’m embarking on a highly simplified explanation that will make specialists cry blood.
The principle of a language model is to look at one part of a sentence and predict the next word. GPT-3 is a technology that has been trained on a SCANDALOUS amount of text written by humans. Nearly a TRILLION words via common crawl. Based on this small sample, it generates text (output) that makes the most sense for an input.
Concretely, by learning to predict which letters and words tend to follow each other when written by a human, GPT-3 is able to predict text that makes sense in a context. It’s the same concept as your cell phone predicting the next word when you write a message. Except that it’s infinitely bigger, more complicated and more advanced.
And so far, GPT-3 is not bringing much new. The architecture of this technology (transformer-based) is the same as the previous version (GPT-2). This architecture is almost 3 years old. So what is so special about GPT-3?
Huge
GPT-3 is absolutely gigantic! GPT-2 was already huge with no less than 1.5 billion parameters. The previous record was held by Turing-NLG of microsoft with these 17 billion parameters. GPT-3 explodes all records with an indecent 175 billion parameters! Never seen before.
And the aftermath of this is impressive results. GPT-3 is capable of far too much.
- write poems in the style of certain authors
- write your emails with your own style
- write blog posts about himself
- write number one blog posts on hacker news
- write whole fictions, many fictions
- even infinite interactive fiction games
- design simple applications
- the list is endless and it’s getting longer every day… if you want to scare yourself, here’s a website that groups them together.
And GPT-3 is capable of something else that immediately worried developers. GPT-3 is capable of producing code. Do you want a todo list application in ReactJS instantly? Just ask for it.
She can produce code but also read and interpret it. She can do this in any language. Some SQL to do ? No problem. CSS? Easy. Even 3D in Javascript can be done. In short, some easy developer tasks can now be automated by GPT-3.
Of course, with all this exploding at the same time on the Internet, many have asked themselves the question of the future of developers. For some it’s the beginning of the end.
As I write this, OpenAI refuses to give away the source code despite its goal of transparency. The official reason? This technology is too big and greedy for people to run it themselves. They put it behind a paywall that allows OpenAI to monetize its research.
Two months after the first appearance of GPT-3, most people agree on one thing. GPT-3 is as superb as overrated. And this is especially true for developers.
Overhyped
GPT-3 is super impressive. It’s a great tool to do a lot of things. But the important thing to understand is that GPT-3 doesn’t understand anything it produces. It makes a mathematical prediction that certain letters should appear in a certain order.
A developer’s job is infinitely more complicated. Developing a complete application that meets a human need requires a lot of thought. No algorithm has the ability to think about problems the way humans do. No thinking, no reasoning ability, no nothing…
Going further in the reflection we realize that GPT-3 is simply a new tool for developers. It is almost a new language where the interface would be directly explanations of what needs to be done instead of code.
And it’s quite relevant to compare the instructions given in GPT-3 to code. GPT-3 doesn’t understand vague requirements. GPT-3 needs very precise, surgical instructions, or it will produce crap. Where a human being will translate, correct and arrange vague requirements, GPT-3 will just produce crap. Or produce nothing at all.
GPT-3 is the largest look-up table ever created. Superintelligence is another story. It’s anything but today, and Sam Altam, the second founder of OpenAI, says so himself.
And all this, not to mention the many limitations and nonsense that GPT-3 produces every day. So right now, developer are safe.
What about tomorrow?
Last year, I posted this article on the future of developers (french version). I’m talking about no-code, low-code, quantum computing and many other nice things. I made the crazy prediction that in 30 years (that’s a lot) the machine will start to nibble away at some parts of our work. No replacement, just a nibbling of certain tasks, or even certain parts.
I had been crucified. Insults by email and on Twitter, unsubscribing and disappointment. I left the comments section as is (as always), it’s a beauty.
A year later, GPT-3 makes its appearance and proves that some very simple developer tasks can already be delegated to the machine.
GPT-3 can’t do anything against the job of developer. But this technology raises the skill bar for becoming a developer very slightly. To do what the machine can’t do on its own.
And if this bar is already rising today, where will it be in 30 years? I’m the first to find the speeches that demonize current IAs ridiculous. But the speeches that talk about an AI as stupid as its feet for eternity seem very naive to me. Where will be the skill bar needed to beat the machine in 30 years?
I am also well aware that the way we make artificial intelligence today is reaching a ceiling. It would take a new and innovative way of doing artificial intelligence to make sense of my concerns. How can we be sure that these new ways of doing things will not emerge in the future?
Epilogue
For now, the future of programming is programming. Artificial intelligence continues its path, but as long as it doesn’t understand what it does, it’s going to be fine. You just have to put up with the fact that it is capable of more and more destabilizing things. How can you be sure that the article you just read is not written by an artificial intelligence?
