OpenAI Dev Day 2023: Four Major Announcements From the Founder Sam Altman’s Keynote You Must Not Miss!

Author(s): Vino Duraisamy

Originally published on Towards AI.

From updates to ChatGPT, the introduction of GPT-4 Turbo, Custom GPTs and GPT store, Assistants API, revised pricing for the models, improved function calling, built-in retrieval, and more.

The first-ever developer conference by OpenAI was packed with amazing product announcements. What’s even more interesting???

These announcements will make a bunch of AI startups totally obsolete. In cricket, we call it “clean bowled”.

Just wiping off so many AI startups and their value prop in an hour long keynote. Isn’t that a dream?! Or a nightmare dare I say!

Dive in to learn more about product announcements and my U+1F336️U+1F336️U+1F336️ takes!!!

1. GPT-4 Turbo

Increased context window

• With an increased context window of 128k, updated knowledge cutoff till April 2023 and the capability to browse the web, it can fit 300 pages of a standard textbook in a single prompt. For a quick comparison, GPT-4 supported an 8k context window. 8k to 128k, this is huge!!!!

JSON Mode in Model Responses

• You can enable JSON mode to request the model to return a JSON response and invoke external application API easily. No more trying to prompt engineering the model to respond with JSON snippets and no other words.
• You can invoke multiple functions from a single prompt.
• Access reproducible model outputs by configuring a seed parameter.
• You can also see the log probabilities of all tokens in a response from the API as well.

Multimodal capabilities

• DALL-E 3 can programmatically generate images
• GPT-4 Turbo with Vision can accept images as inputs via the API
• TTS Audio API that provides a Text-to-Speech endpoint, with 6 built-in voices. And it sounds very natural
• Whisper V3, the open-source Automatic Speech Recognition Model, was announced. It will be available in the API soon, Altman said.

Model Fine-tuning

• Foundation model fine-tuning is as much an art as it is science. Or dare I call it a witchcraft?
• The OpenAI team will work with companies to help them fine-tune GPT models for specific domains and datasets.
• Experimental access to GPT-4 fine-tuning is open too. Sign up if you are looking to get your hands on that.

Copyright Shield and Rate limits

• OpenAI doubled the tokens per minute for all GPT-4 users
• Copyright Shield is to protect the GPT-4 customers on the legal front if they run into copyright issues by the usage of GPT models. While the creative liberties of artists and the copyright laws need a revamp thanks to AI, this is an interesting take and let’s see how this evolves as the field matures

Pricing

• Apparently GPT-4 Turbo will be 2.75x cheaper on average, encouraging users to use this model as opposed to the previous ones.

2. Updates to chatGPT

• chatGPT now uses GPT-4 Turbo
• It can browse the web through Bing.
• While chatting, you don’t need to select the model using the dropdown. Based on the input prompt, chatGPT knows which model to invoke and when.

3. GPTs and GPT store

• GPTs are the tailored versions of chatGPT for different domains and use cases.
• These are the evolution of chatGPT plug-ins. You can build a custom version of chatGPT with instructions, expanded knowledge retrieval and actions.
• The OpenAI team woke up and chose violence. I mean, not in the literal sense. Or maybe they did. How many startups went out of business today?! Should we keep count?
• Prompt Engineering, Retrieval Augmented Generation and building AI apps by chaining different API calls using LangChain or similar libraries were the most popular approaches to building AI apps.

With expanded knowledge retrieval, improved function calling, and Assistants API from OpenAI, a slew of tools in the so called “LLM stack” just ran out of their value prop. The pace at which the industry is moving, unimaginable!!!! Super scary and equally exciting!!

• GPT Builder lets you write input prompts in natural language that generate customGPTs for you. No more prompt engineering, dealing with embeddings and vector databases, writing code to create a chatbot-like experience.
• One AI agent creates multiple AI agents. Well, well, we are on the way to building an AI factory lol.
• TutorGPT, WriterGPT, ResumeGPT, ThisGPT, ThatGPT, and every other tool that was a wrapper around GPT can now be built with natural language. Wild, isn’t it?!

GPT store

• After building the custom GPTs, you can publish it for folks to use in the GPT store. It’s an online marketplace for different apps and would be a great resource for companies that don’t have AI experts to build custom solutions for them.
• This will make AI adoption to the last mile super easy. Even the long tail of companies that are at a disadvantage due to a lack of AI experts in-house can benefit in a massive way.
• Revenue sharing with GPT creators will mean there may be a swarm of one-person AI companies raking big bucks. Are one-person AI tech unicorns going to be a thing now? Or am I trusting OpenAI’s revenue-sharing plan too much? Let’s wait and watch!!!!
• Oh wait! What about the safety of the GPTs that are published in the store? Altman says there will be a rigorous vetting process to publish these GPTs in the store.
• But what of the existing versions of GPT, such as ChaosGPT?

4. Assistants API

• Assistants API makes building chatbots and agent-like experiences straightforward. No more haggling with embeddings, vector databases, chaining different API calls using a separate tool, relying on RAG(Retrieval Augmented Generation) architecture, etc.
• One of the biggest use cases of LLM models is to build chatbots for different domains.
• Assistants API supports persistent threads, so more state management, prompt and context management hassles.

It comes with built-in retrieval to augment the model with external knowledge repository. You don’t need to compute the embeddings for the knowledge base documents, no more storing them in a vector database or having to implement a smart chunking algorithm. How freaking cool, right? Don’t need no RAG anymore.

• In the past few months, every tech company that you ever know has released its own LLM assistant and calls them all sorts of fancy two-syllable English names. Unless you are Microsoft and chatGPT knows what that chatbot will be named U+1F602U+1F602U+1F602

Python Interpreter

• Remember in the initial days of chatGPT-mania, some of us thought chatGPT must have a linux VM inside of it? Because when we ran a few Linux commands in the input prompt, it generated unbelievably correct results to those commands. The only possible explanation was that chatGPT ran our commands on the VM and returned the results. How else is a large language model going to return perfect results on Linux commands?!!!

Well, the OpenAI team caught up to our whimsies and decided to put in a code interpreter that can run Python code in a sandboxed execution environment. It can plot charts, perform data analysis, generate python code to solve math and reasoning problems and what not.

• This is so cool, but will also fuel more questions/research in the “Oh chatGPT can actually do reasoning now” direction.
• The keynote demo shows how chatGPT was able to do simple math to split the expenses between friends, convert exchange rates and calculate the total expenses for a trip to Paris. But can it solve the Blocks world? Well, it’s not a me problem, it’s a you problem (looking at you Symbolic AI researchers!!U+1F605)

Summary

Assistants API is a game changer for organizations and GPT store is a game changer for builders. Let’s go build!

You can watch the full keynote as well.

Thanks for Reading!

If you like my work and want to support me…

1. The BEST way to support me is by following me on Medium.
2. For data engineering best practices, and Python tips for beginners, follow me on LinkedIn.
3. Feel free to give claps so I know how helpful this post was for you.

Stackademic

Thank you for reading until the end. Before you go:

• Please consider clapping and following the writer! U+1F44F
• Follow us on Twitter(X), LinkedIn, and YouTube.
• Visit Stackademic.com to find out more about how we are democratizing free programming education around the world.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI