Monday, April 12

GPT-3: the model that enables computers to write stories

Machine learning models are pushing the boundaries of what they can achieve in a monthly basis. This time, it is not virtual scenery or autonomous cars, it is about text generation, be it conversation, article writing or technical manuals. GPT-3 is created by OpenAI and is the latest and most powerful language model ever created. Just over a year earlier, its predecessor, GPT-2 was released which could already create syntactically correct sentences. The leap to the current version is huge: the neural network in GPT-2 had 1.5 billion parameters, GPT-3 has 175 billion!

Due to the disruptive potential of this technology (discussed next), the model is not publicly available and OpenAI has released an API (like a webapp) for selected people to experiment with it without giving away the actual parameters. I recommend you to have a look at this blog post by Arram Sabeti where he shares a songs, fictional news and even a technical manual, all created by giving the AI only the title. As you can see, the model successfully used the appropriate language for each situation; what’s more scary, it can even attempt to imitate the way celebrities speak.

Screenshot of GPT-3 Playground

Can you think what will be the implications of this technology? These could be some negative consequences:

  • Manipulation of the dialogue in social networks. In the past there have been manipulation campaigns in Twitter where bots try to flood hashtags with repetitive comments; these are easy to detect. However new synthetic texts will be hard to spot and could be used to give users an illusion that the majority opinion is one that isn’t real.
  • Increase in fake news. Combined with videofakes and audiofakes, complete ‘backed-up’ news articles could cause scandals on targeted people. With the correct timing, it could change the elections outcome or bring down governments.
  • Newspapers could layoff a part of their writers. Economic events, press conferences, celebrity announcements, etc. all happen online meaning that journalists no longer need to travel and the writing can be done by GPT-3.
  • Decrease in revenue for freelance content writers. Millions of people earn a living from writing for their blogs or for clients. A person con only write so much, so the revenue is well distributed. In the future, the majority of content could be synthetically generated leaving human writers with a tiny share.

The fact that this technology was created by a company partly created to make advances in AI accessible to everyone is slightly encouraging because it places public research institutions and corporations on a level playing field. It must also serve to know what is the current state of the art in synthetic text. We need to be aware that soon bad actors could create millions of fake news articles and online conversation to manipulate our opinions in very little time.

Leave a Reply

Your email address will not be published. Required fields are marked *