I Wasted 6 Months Building AI Models Before Learning These 7 Frameworks

Author(s): DefineWorld

Originally published on Towards AI.

It was 2 AM. My fourth cup of coffee sat cold on my desk. I had just spent three weeks building a custom training loop for a computer vision model… only to discover that someone had already built a framework that did exactly what I needed… in about 47 lines of code.

That hurt.

Not because I wasted three weeks… but because I realized how many other things I had probably been overcomplicating. The truth about AI development that nobody tells you when you are starting is this: knowing which frameworks exist is more valuable than knowing how to build everything from scratch.

Let me save you from my mistakes. These seven frameworks changed how I build AI systems….. and honestly….. some of them feel like cheating.

LangChain….. When You Stop Fighting With LLM APIs

Remember when integrating language models into your application meant writing hundreds of lines of API wrapper code?

I do. I spent two months last year building a document question answering system for a client. The code was a mess. API calls everywhere. Prompt management is scattered across different files. No way to swap between different models without rewriting half the codebase.

Then I found LangChain.

It is not just an API wrapper. It is a complete framework for building LLM-powered applications. Chains for sequential operations. Agents that can use tools. Memory systems that actually work. The ability to switch between OpenAI….. Anthropic… or local models without changing your core logic.

Here is something I built last week that would have taken me days before…..

from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser
from langchain.schema.runnable import RunnablePassthrough
from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.document_loaders import TextLoader

# Load and process documents
loader = TextLoader("knowledge_base.txt")
documents = loader.load()
text_splitter = RecursiveCharacterTextSplitter(
 chunk_size=1000,
 chunk_overlap=200
)
splits = text_splitter.split_documents(documents)
# Create vector store
vectorstore = FAISS.from_documents(
 documents=splits,
 embedding=OpenAIEmbeddings()
)
retriever = vectorstore.as_retriever()
# Build RAG chain
template = """Answer the question based only on the following context.....
{context}
Question..... {question}
"""
prompt = ChatPromptTemplate.from_template(template)
model = ChatOpenAI(model="gpt-4")
chain = (
 {"context": retriever, "question": RunnablePassthrough()}
 | prompt
 | model
 | StrOutputParser()
)
# Use it
response = chain.invoke("What are the main topics discussed?")
print(response)

That is a complete RAG system. Vector storage. Document retrieval. LLM integration. All in one clean pipeline.

The best part? Swap out ChatOpenAI for ChatAnthropic or any other model….. and everything still works.

Hugging Face Transformers….. The Library That Democratized AI

I need to be honest about something… before Transformers….. I was terrified of working with pre-trained models.

The documentation was scattered. Every model had a different API. Loading weights felt like performing surgery. Fine-tuning required a PhD in PyTorch internals.

Hugging Face changed everything.

Over 150,000 pre-trained models. One consistent API. Download any model with a single line. Fine-tune with minimal code. Deploy to production without wanting to quit your job.

Here is how simple it is to use a state-of-the-art model now…..

from transformers import pipeline
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Method 1..... Ultra simple pipeline
classifier = pipeline("sentiment-analysis")
result = classifier("I love building AI applications!")
print(result)
# Method 2..... More control
model_name = "distilbert-base-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
# Tokenize and predict
inputs = tokenizer("This framework is incredible", return_tensors="pt")
with torch.no_grad():
 outputs = model(**inputs)
 predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
 
print(f"Positive..... {predictions[0][1].item():.3f}")
print(f"Negative..... {predictions[0][0].item():.3f}")
# Fine-tuning is just as simple
from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
 output_dir="./results",
 num_train_epochs=3,
 per_device_train_batch_size=16,
 warmup_steps=500,
 weight_decay=0.01,
 logging_dir="./logs",
)
trainer = Trainer(
 model=model,
 args=training_args,
 train_dataset=train_dataset,
 eval_dataset=eval_dataset
)
trainer.train()

I used this exact pattern last month to fine-tune a model for technical document classification….. something that would have taken me weeks of custom PyTorch code two years ago….. done in an afternoon.

LlamaIndex….. Because RAG Should Not Be This Hard

You know what frustrated me most about building retrieval systems? Everything.

Chunking strategies that made no sense. Vector databases that required a computer science degree to configure. Query engines that could not handle follow-up questions. The code always turned into spaghetti.

LlamaIndex solved problems I did not even know I had.

It handles document loading from anywhere….. PDFs….. websites….. databases….. whatever. Smart chunking that actually preserves context. Multiple indexing strategies. Query engines that feel magical. And the ability to combine different data sources seamlessly.

from llama_index import VectorStoreIndex, SimpleDirectoryReader
from llama_index import ServiceContext
from llama_index.llms import OpenAI
from llama_index.embeddings import OpenAIEmbedding

# Load documents from a directory
documents = SimpleDirectoryReader("data").load_data()
# Configure the service
service_context = ServiceContext.from_defaults(
 llm=OpenAI(model="gpt-4", temperature=0.1),
 embed_model=OpenAIEmbedding()
)
# Create index
index = VectorStoreIndex.from_documents(
 documents,
 service_context=service_context
)
# Query with context awareness
query_engine = index.as_query_engine(
 similarity_top_k=3,
 response_mode="tree_summarize"
)
response = query_engine.query(
 "What are the key findings from the research papers?"
)
print(response)
# Follow-up questions just work
response2 = query_engine.query(
 "Can you elaborate on the methodology used?"
)
print(response2)
# Want to combine multiple indexes? Easy.....
from llama_index.composability import ComposableGraph
graph = ComposableGraph.from_indices(
 SimpleKeywordTableIndex,
 [index1, index2, index3],
 index_summaries=["Research papers", "Technical docs", "Blog posts"]
)
query_engine = graph.as_query_engine()

The first time I used this for a client project, they asked if I had hired a team. Nope… just stopped fighting with custom implementations.

Gradio….. Making AI Demos That Do Not Embarrass You

Here is something nobody talks about… most AI developers are terrible at building interfaces.

I know this because I was one of them. My demos looked like they were built in 1995. Command line outputs. No error handling. Zero visual appeal. And getting stakeholders to test things? Forget it.

Gradio changed my demo game completely.

Web interfaces with a few lines of code. Automatic API generation. Built-in examples. File uploads that actually work. And it looks professional without any CSS nightmares.

import gradio as gr
from transformers import pipeline

# Load your model
sentiment_analyzer = pipeline("sentiment-analysis")
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
def analyze_text(text):
 sentiment = sentiment_analyzer(text)[0]
 return f"Sentiment..... {sentiment['label']} (confidence..... {sentiment['score']:.3f})"
def summarize_document(text):
 if len(text.split()) < 50:
 return "Text too short to summarize"
 summary = summarizer(text, max_length=130, min_length=30, do_sample=False)
 return summary[0]['summary_text']
# Create interface with multiple functions
with gr.Blocks() as demo:
 gr.Markdown("# AI Text Analysis Tool")
 
 with gr.Tab("Sentiment Analysis"):
 text_input = gr.Textbox(
 label="Enter text to analyze",
 placeholder="Type something here...",
 lines=3
 )
 sentiment_output = gr.Textbox(label="Result")
 sentiment_btn = gr.Button("Analyze Sentiment")
 sentiment_btn.click(
 fn=analyze_text,
 inputs=text_input,
 outputs=sentiment_output
 )
 
 with gr.Tab("Summarization"):
 doc_input = gr.Textbox(
 label="Enter document to summarize",
 placeholder="Paste your document here...",
 lines=10
 )
 summary_output = gr.Textbox(label="Summary")
 summary_btn = gr.Button("Generate Summary")
 summary_btn.click(
 fn=summarize_document,
 inputs=doc_input,
 outputs=summary_output
 )
 
 gr.Examples(
 examples=[
 ["This product exceeded my expectations!"],
 ["Terrible experience, would not recommend."]
 ],
 inputs=text_input
 )
demo.launch(share=True)

That code creates a professional-looking web interface….. complete with tabs….. examples….. and a shareable link. I have used this pattern for every client demo in the past year.

Weights & Biases….. When You Stop Losing Experiment Results

I lost three weeks of experiment results once.

Not because my hard drive crashed. Not because of a bug. But because I forgot which hyperparameters produced which results….. and I had been tracking everything in a text file like an animal.

Never again.

Weights & Biases tracks everything automatically. Every metric. Every hyperparameter. Every model checkpoint. Visualizations that actually help you understand what is happening. And the ability to compare hundreds of experiments without losing your mind.

import wandb
from transformers import Trainer, TrainingArguments
from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Initialize tracking
wandb.init(
 project="text-classification",
 config={
 "learning_rate": 2e-5,
 "epochs": 3,
 "batch_size": 16,
 "model_name": "distilbert-base-uncased"
 }
)
config = wandb.config
# Load model
model = AutoModelForSequenceClassification.from_pretrained(
 config.model_name,
 num_labels=2
)
# Training arguments with W&B integration
training_args = TrainingArguments(
 output_dir="./results",
 num_train_epochs=config.epochs,
 per_device_train_batch_size=config.batch_size,
 learning_rate=config.learning_rate,
 logging_steps=10,
 evaluation_strategy="steps",
 eval_steps=50,
 save_steps=100,
 report_to="wandb"
)
trainer = Trainer(
 model=model,
 args=training_args,
 train_dataset=train_dataset,
 eval_dataset=eval_dataset
)
# Train and automatically log everything
trainer.train()
# Log custom metrics
wandb.log({
 "custom_metric": some_value,
 "validation_accuracy": accuracy
})
# Save model artifact
wandb.save("model.pth")
wandb.finish()
# Compare experiments later with W&B dashboard

The dashboard shows me every experiment I have ever run. Filter by hyperparameters. Compare loss curves. Find the best performing model in seconds.

I wish I had discovered this five years ago.

Streamlit….. Because Production UIs Should Not Take Months

Building production interfaces used to mean learning React….. setting up backend APIs….. dealing with state management….. and questioning my career choices.

Streamlit lets you skip all of that.

Pure Python. No JavaScript. No HTML. No CSS. Just write your logic and get a working web app. And it does not look like a prototype… it looks production-ready.

import streamlit as st
from transformers import pipeline
import pandas as pd

st.set_page_config(page_title="AI Analysis Dashboard", layout="wide")
st.title("Real-time AI Analysis Dashboard")
# Sidebar configuration
with st.sidebar:
 st.header("Configuration")
 model_choice = st.selectbox(
 "Select Model",
 ["distilbert-base-uncased", "roberta-base", "albert-base-v2"]
 )
 confidence_threshold = st.slider(
 "Confidence Threshold",
 0.0, 1.0, 0.5
 )
# Load model with caching
@st.cache_resource
def load_model(model_name):
 return pipeline("sentiment-analysis", model=model_name)
classifier = load_model(model_choice)
# Main interface
tab1, tab2 = st.tabs(["Single Analysis", "Batch Analysis"])
with tab1:
 user_input = st.text_area("Enter text to analyze", height=150)
 
 if st.button("Analyze"):
 if user_input:
 with st.spinner("Analyzing..."):
 result = classifier(user_input)[0]
 
 col1, col2 = st.columns(2)
 with col1:
 st.metric("Sentiment", result['label'])
 with col2:
 st.metric("Confidence", f"{result['score']:.3f}")
 
 if result['score'] < confidence_threshold:
 st.warning("Low confidence prediction")
 else:
 st.error("Please enter some text")
with tab2:
 uploaded_file = st.file_uploader("Upload CSV file", type=['csv'])
 
 if uploaded_file is not None:
 df = pd.read_csv(uploaded_file)
 st.write("Preview of data.....") 
 st.dataframe(df.head())
 
 if st.button("Analyze All"):
 results = []
 progress_bar = st.progress(0)
 
 for idx, text in enumerate(df['text']):
 result = classifier(text)[0]
 results.append(result)
 progress_bar.progress((idx + 1) / len(df))
 
 df['sentiment'] = [r['label'] for r in results]
 df['confidence'] = [r['score'] for r in results]
 
 st.success("Analysis complete!")
 st.dataframe(df)
 
 st.download_button(
 "Download Results",
 df.to_csv(index=False).encode('utf-8'),
 "analysis_results.csv",
 "text/csv"
 )

I built a complete analytics dashboard with this in two hours. My manager thought I had been working on it for weeks.

FastAPI with ML….. Deploying Models Without The Pain

The last framework….. and maybe the most important….. is not AI-specific. But it changed how I deploy models.

Flask was fine. Until I needed async support. Or automatic API documentation. Or data validation. Or WebSocket support. Or any of the hundred other things modern ML APIs need.

FastAPI does all of that… and it is faster than anything else in Python.

from fastapi import FastAPI, File, UploadFile, HTTPException
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from transformers import pipeline
import uvicorn
from typing import List
import numpy as np

app = FastAPI(
 title="AI Model API",
 description="Production-ready ML model serving",
 version="1.0.0"
)
# Load model at startup
@app.on_event("startup")
async def load_models():
 global classifier, summarizer
 classifier = pipeline("sentiment-analysis")
 summarizer = pipeline("summarization")
# Request models with validation
class TextInput(BaseModel):
 text: str
 max_length: int = 130
 
class BatchInput(BaseModel):
 texts: List[str]
# Endpoints with automatic documentation
@app.post("/analyze/sentiment")
async def analyze_sentiment(input_data: TextInput):
 try:
 result = classifier(input_data.text)[0]
 return {
 "text": input_data.text,
 "sentiment": result['label'],
 "confidence": float(result['score'])
 }
 except Exception as e:
 raise HTTPException(status_code=500, detail=str(e))
@app.post("/analyze/batch")
async def batch_analyze(input_data: BatchInput):
 try:
 results = []
 for text in input_data.texts:
 result = classifier(text)[0]
 results.append({
 "text": text,
 "sentiment": result['label'],
 "confidence": float(result['score'])
 })
 return {"results": results}
 except Exception as e:
 raise HTTPException(status_code=500, detail=str(e))
@app.post("/summarize")
async def summarize_text(input_data: TextInput):
 try:
 if len(input_data.text.split()) < 50:
 raise HTTPException(
 status_code=400,
 detail="Text too short for summarization"
 )
 
 summary = summarizer(
 input_data.text,
 max_length=input_data.max_length,
 min_length=30,
 do_sample=False
 )
 return {
 "original_text": input_data.text,
 "summary": summary[0]['summary_text']
 }
 except HTTPException:
 raise
 except Exception as e:
 raise HTTPException(status_code=500, detail=str(e))
@app.get("/health")
async def health_check():
 return {"status": "healthy", "models_loaded": True}
if __name__ == "__main__":
 uvicorn.run(app, host="0.0.0.0", port=8000)

Run that code, and you get automatic interactive documentation at /docs….. request validation….. error handling….. and an API that can handle thousands of requests per second.

I deployed a model to production last week using this exact pattern. From local testing to production in one day.

The Pattern Nobody Tells You About

Here is what I learned after wasting six months building everything from scratch…..

The best AI developers are not the ones who can implement every algorithm. They are the ones who know which frameworks exist… and how to combine them effectively.

LangChain for LLM orchestration. Transformers for model access. LlamaIndex for retrieval. Gradio for quick demos. Streamlit for production UIs. Weights & Biases for experiment tracking. FastAPI for deployment.

Each one solves a specific problem better than you ever could by yourself.

Use them. Combine them. Build things that would have been impossible a year ago.

And stop wasting time reinventing wheels that are already rolling.

So here is my question for you… Which framework are you going to try first? And what AI project have you been putting off because you thought it would be too complicated?

Drop your answer below. Let us see what problems we have all been overcomplicating.

If you enjoyed reading, be sure to give it 50 CLAPS! Follow and don’t miss out on any of my future posts — subscribe to my profile for must-read blog updates!

Thanks for reading!

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI