Modern machine learning and AI research have moved rapidly from the lab to our IDEs, with tools like Azure’s Cognitive Services providing API-based access to pretrained models. There are many different approaches to delivering AI services, with one of the more promising methods for working with language being a technique called generative pretraining or GPT, which handles large amounts of text.
OpenAI and Microsoft
The OpenAI research lab pioneered this technique, publishing the initial paper on the topic in 2018. The model it uses has been through several iterations, starting with the unsupervised GPT-2, which used untagged data to mimic humans. Built on top of 40GB of public internet content, GPT-2 required significant training to provide a model with 1.5 billion parameters. It was followed by GPT-3, a much larger model with 175 billion parameters. Exclusively licensed to Microsoft, GPT-3 is the basis for tools like the programming code-focused Codex used by GitHub Copilot and the image-generating DALL-E.
To read this article in full, please click here
InfoWorld Cloud ComputingRead More
