Chord Labs | Get Started Guide: Custom GPT AI/ML App Development

Author(s): Xiao Zhou

Originally published on Towards AI.

Chord Labs U+007C Get Started Guide: Custom GPT AI/ML App Development

4-Step Plan to Build a GPT / ChatGPT App from Idea to Production

API-accessed models like OpenAI’s GPT are making machine learning app development easy, at least for Natural Language Processing use cases. Think chatbots.

So let’s talk about how to go from idea to production release. Avoid being one of the ~80% of AI/ML projects that never make it.

Our team followed 4 steps to build our GPT-powered machine learning app called /takeaway, which returns key action items and topic summaries from Slack channels. The project took about 3 months end to end.

1. Define a problem to solve

Starting with a clear problem statement will ensure the app does something useful.

For /takeaway, the problem was one my partner and I both faced working remotely: too many chat messages and not enough time to read them. We constantly asked ourselves: “Can I skip this thread? What do they need me to do? What did I forget to follow up on?”

So our problem statement became:

1. Ideal: Users can quickly see the gist of chat threads, choose whether to read their entirety and also be reminded later on of what was talked about during the day. This allows users to prioritize their time while staying informed.
2. Current reality: The flood of chat messages has become completely unmanageable for users. Despite their best efforts to keep up, users are struggling to choose which chat threads to read. Most users attempt to read all of the messages while also trying to focus on other important work, leading to a constant state of distraction and overwhelm. As a result, crucial tasks and deadlines are sometimes missed, and important information is overlooked.
3. Consequences: Missed deadlines and overlooked tasks can lead to decreased productivity, damaged relationships, and potentially even financial losses. Additionally, feeling overwhelmed and disorganized can lead to stress and burnout, which can have negative impacts on both personal and professional well-being.
4. Proposal: A solution is to use machine learning to summarize and surface action items from chat threads. Visibility into topics of long chat threads allows users to choose whether to read them. Additionally, users can remind themselves of key actions to follow up on at the end of a long day. Avoid forgetting key takeaways in the chaos of virtual work.

2. Research if AI/ML is the right solution

We can’t eat soup with a knife.

Similarly, AI/ML are not solutions to everything. For example, manufacturing efficiency is best optimized via lean methodology, and imagining a new business strategy is best left to a creative human brain.

To check if AI/ML is a relevant solution, it helps to research how a problem has been historically solved. Has someone already used machine learning to solve a similar problem?

For /takeaway, our research showed that a specific field of machine learning known as Natual Language Processing (NLP) has produced many methods to summarize topics from text. We also discovered that OpenAI-GPT3 has trained specific models to summarize text data.

So AI/ML might just be the solution to our problem. We just had to see how it’d perform with chat data.

3. Design a proof of concept, cheaply

It’s wise to build a working prototype to ensure that the solution can work as intended.

Luckily for us, OpenAI offers a “playground” to test its different models, such as its text classifier or ChatGPT engines. So we found and loaded into GPT a series of chat texts from publicly available sources like NPR and some messages from the Enron Email Dataset, and also made-up imaginary conversations.

Our tests showed that GPT was capable of summarizing topics and surfacing actions out of the box. Roughly 50% of results returned by GPT-3 appeared relevant. The engine did have tendencies to make stuff up. It was especially bad at interpreting emojis, where a ghost emoji was interpreted as murder. Scary stuff.

We proceeded to fine tuned a unique GPT-3 model to improve relevancy. After some iterations, /takeaway yielded relevant results ~90% of the time. And no more murders. That was good enough for V1.

4. Build a production version

We began developing everything around /takeaway’s summary engine once we had confidence in GPT’s ability to summarize chat data. This meant:

• Integration with Slack: We set up a test Slack account and learned the ins and outs of its API. Slack is very much trying to build an app ecosystem, so it was quite straightforward to build an app for it. We were able to extract chat data within about a day. Finishing building the app end-to-end, testing, and approval took many more weeks. Following the Slack App Directory guidelines helped speed things up during the review with the Slack team.
• Designing a user experience: We created a user flow aiming for a minimalistic experience. We also tried to identify all edge use cases and error messages needed. A key design decision was to default the /takeaway command to only pull data from the 24 hours preceding a channel’s latest note, so it would provide timely insights and not overwhelm users with a big thread of its own.
• Fixing key bugs: Finally, we set up logging to help debug. We asked a few friends to test the app and found bugs with the installation flow, data extraction from Slack and results readability. Every new bug discovered allowed us to improve both the app and the logging.

Step #4 could take much longer, depending on the app’s complexity. We decided not to integrate any payment processing systems to keep development costs down, so the app’s free.

What’s next? We’ve got a few ideas to try with the new ChatGPT API, so stay tuned.

We hope this helps inspire more folks to build AI/ML apps with GPT. Drop me a note to partner up on a project or integrate GPT into your app 🙂

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI