Why OpenAI’s o1 Model Is A Scam

Author(s): Artem Shelamanov

Originally published on Towards AI.

As a data scientist who has worked with LLMs since they were first introduced, I thought that when I heard about o1, it was a joke. When it turned out to be an actual, real model, I thought I had misunderstood something. But after reading through the documentation, I knew what it was — a marketing ploy to get more money.

This may sound controversial, but bear with me — let’s dive into the technical details, analyze how the o1 works and why exactly you shouldn’t pay for it.

How o1 Works

As we all know, OpenAI is not so open as usual, revealing almost no technical details about how o1 works. From their official release page, all the information we got is:

We trained these models to spend more time thinking through problems before they respond, much like a person would. Through training, they learn to refine their thinking process, try different strategies, and recognize their mistakes.
…
…for complex reasoning tasks this is a significant advancement and represents a new level of AI capability. Given this, we are resetting the counter back to 1 and naming this series OpenAI o1.

Let’s write down the main conclusions:

1. o1, unlike previous models, tries to “think” before writing the final answer.
2. The thinking process is based on the “Chain of Thought” technique — it thinks step by step, tries different methods, recognises mistakes, and only when the final result is ready and works does the model send it to the user.

Why It’s A Scam

To start off, look at the following photos:

These are just two examples. Basically, people were trying to ask o1 about its own CoT (Chain of Thought) — they just wanted to know how it worked.

This raises suspicions. Why would OpenAI not only hide technical details about CoT, but also ban people from trying to understand how it works?

The answer is simple, and LLM enthusiasts know it — this technique has been used for years. It wasn’t a “new approach” introduced by the OpenAI team; it was just a copy-paste + marketing strategy.

How CoT Works

If you have access to the OpenAI API (or any other LLM model, really), you can have your own CoT working in less than 30 lines of code. If you use libraries like LangChain, it’s even easier. But let’s implement it from scratch to qualify as OpenAI senior researchers.

First, pip-install openai library:

pip install openai

After that, you create a .py file with the following content and run it:

from openai import OpenAI
import json

client = OpenAI()
prompt = """
You are an AI agent that thinks before answering.
Answer in the JSON format. Respond only with the json, no other text.
Example:
{
 "step1": "To solve this problem, i need to do this...",
 "step2": "Based on results from step1, i need to do this...",
 ...
 "stepn": "We can get final result by...",
 "result": "Final result is..."
}
Your question is: write python code to get the first 10 prime numbers.
"""
completion = client.chat.completions.create(
 model="gpt-4o-mini",
 messages=[
 {
 "role": "user",
 "content": prompt
 }
 ]
)
result_json_str = completion.choices[0].message.content
result_json = json.loads(result_json_str.replace("```json", "").replace("```", ""))
print(result_json)

The result is the following JSON:

{
 "step1": "To solve this problem, I need to define a function that checks if a number is prime.",
 "step2": "I will iterate through numbers starting from 2 and check each one for primality.",
 "step3": "I will keep track of the found prime numbers in a list until I have 10 primes.",
 "step4": "I will return the list of the first 10 prime numbers.",
 "result": "The Python code is:
 python
 def is_prime(num):
 if num <= 1:
 return False
 for i in range(2, int(num**0.5) + 1):
 if num % i == 0:
 return False
 return True
 primes = []
 num = 2
 while len(primes) < 10:
 if is_prime(num):
 primes.append(num)
 num += 1
 print(primes)
 "
}

Using this JSON, you can get your final result from the “result” field, and intermediate steps from “step1”, “step2”, and so on. The only thing left to do is to create a pretty frontend for it, and whoila — you have your very own “o1”. Now you can proudly say “Don’t pay for o1 — I have it at home”.

Some technical details: when LLM generates a new token (basically the next word), it uses all the previous text as input. This means that when ChatGPT generates a reasoning step, it sees every previous step it took, and as a result we get step-by-step reasoning.

If you want to dive deeper and exceed CoT limitations, I recommend looking into LLM agents and ReAct — these methods allow you to create LLM-based agents that not only think before they respond, but can also do whatever you give them the freedom to do — use google, get the weather, read a file, and just about anything else you want.

Why o1 May Not Be A Scam

One thing I didn’t mention in this article that makes o1 look less like a scam is the fact that OpenAI trained the o1 model using reinforcement learning not only for the final result, but also for the intermediate steps.

In simple words, when we use CoT with simple GPT4, we ask it to think — which it didn’t do before. But when we use o1, the model knows how to think from the start — it has been trained to do so. This means that the performance of o1 may be better than our “o1 at home”. But is it cheaper? And does it give you as much freedom? Who knows?

Summary

In this article, we have analyzed the new OpenAI model, looked deeply into its drawbacks, and learned some AI coding with Python.

As a final conclusion, I just want to say one thing: think before you act. Like o1, you can come to better conclusions by carefully analyzing the information you have. If you see that some big company has released a new feature, don’t rush to pay them for it; take everything with a grain of salt.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI