What is an “AI developer”?

An AI developer builds software that uses machine learning or large language models as a core capability—not only notebooks in a research lab, and not only clicking through a chat UI. You integrate models into products: APIs, backends, agents, data pipelines, and the operational glue (auth, logging, scaling, cost controls) that makes them reliable in production.

Related roles overlap; knowing the difference saves confusion:

• Data scientist — explores data, hypotheses, and models; heavy on statistics and experimentation.
• ML engineer — trains, evaluates, versions, and serves models; bridges data science and platform.
• MLOps / platform engineer — pipelines, registries, GPUs, monitoring, reproducibility at scale.
• AI application developer — product features on top of APIs (RAG, agents, copilots) with strong software craft.
• AI researcher — advances algorithms and publishes; often needs PhD-level math.

Many of us wear several hats. If you already write backend or platform code, you are closer than you think: AI development is still development, with extra moving parts.

Why AI—and why now?

Artificial intelligence is not hype alone. Three forces converged:

• Data and compute — cloud GPUs, cheap storage, and open datasets made training large models feasible.
• Algorithms — deep learning, then transformers, unlocked language, vision, and multimodal tasks at useful quality.
• Distribution — APIs and open weights put capable models in every IDE, browser, and enterprise workflow.

Why it matters for developers:

• Every product layer is touched — search, support, codegen, analytics, security, and operations all gain AI-assisted paths.
• Skills compound — Python, APIs, vectors, evaluation, and cloud patterns transfer across employers and stacks.
• Demand is structural — organizations need people who can ship AI features safely, not only experiment in demos.

The future is not “AI replaces all developers.” It is developers who use AI well outperform those who ignore it—and teams that govern AI responsibly win trust. Expect smaller specialized models, more on-device inference, agentic workflows with human oversight, and regulation (e.g. ISO/IEC 42001-style AI management) as normal engineering constraints. For historical context on how we got here, see From Symbols to Foundation Models.

Core terminology (glossary)

Learn these terms once; they appear in every architecture diagram and job description. For a fuller reference with more terms and confused pairs, see AI and ML terminology.

Learning and models

Artificial intelligence (AI)

Broad field: systems that perform tasks associated with human intelligence (reasoning, perception, language, planning).

Machine learning (ML)

Systems that improve from data without explicit rules for every case.

Deep learning

ML using neural networks with many layers; dominant in vision and language today. Technical depth: Neural Networks in Depth.

Foundation model

Large model pre-trained on broad data, then adapted (fine-tuned, prompted, or RAG-augmented) for tasks. See AI Foundation Models in Depth for architecture, training, and production detail.

Large language model (LLM)

Foundation model for text (and often code); predicts tokens; powers chat and agents. Technical depth: Large Language Models in Depth.

Generative AI

Models that create new content (text, images, audio, video) rather than only classifying inputs.

Training

Adjusting model weights on datasets; expensive, offline, needs GPUs for large models.

Inference

Running a trained model to get predictions or generations; what users hit in production.

Fine-tuning

Further training on domain-specific data to specialize behavior.

Embedding

Dense vector representation of text/images used for similarity search and RAG.

Generative AI and applications

Prompt

Input instructions and context sent to a model; engineering prompts is a real skill.

Token

Subword unit models process; billing and context limits are often token-based.

Context window

Maximum tokens the model can consider in one request.

RAG (retrieval-augmented generation)

Fetch relevant documents from a vector store, inject into the prompt, then generate—grounds answers in your data. See RAG in Depth for chunking, hybrid search, evals, and production patterns.

Agent

LLM loop that plans, calls tools (APIs, DB, code), and iterates toward a goal.

Hallucination

Confident but incorrect output; mitigated with RAG, citations, evals, and human review.

Grounding

Tying responses to verified sources (docs, DB rows, tool results).

Metrics and quality

Loss

Training objective the model minimizes; lower is better during training.

Accuracy / precision / recall / F1

Classic classification metrics; still used for structured ML tasks.

BLEU, ROUGE, etc.

Automated scores for text similarity; weak alone for LLM quality.

LLM-as-judge / human eval

Rating outputs for helpfulness, safety, and faithfulness—essential for Gen AI products.

Operations and governance

MLOps

CI/CD, versioning, monitoring, and deployment practices for ML systems.

Model registry

Catalog of trained artifacts with metadata and lineage.

Feature store

Reusable, consistent inputs for training and serving.

GPU / TPU

Accelerators for training and heavy inference.

Responsible AI

Fairness, privacy, safety, transparency, and compliance—built into design, not a slide deck.

AI system architecture (how pieces fit)

Production AI is a stack of layers. You do not need to master every layer on day one, but you should know where your work sits.

Reference architecture

┌─────────────────────────────────────────────────────────────┐
│  Application (UI, API, agents, business logic)              │
├─────────────────────────────────────────────────────────────┤
│  Orchestration (LangChain, LlamaIndex, custom workflows)    │
├─────────────────────────────────────────────────────────────┤
│  Model access (OpenAI, Anthropic, Bedrock, Ollama, vLLM)    │
├─────────────────────────────────────────────────────────────┤
│  Knowledge (vector DB, search, caches, feature store)         │
├─────────────────────────────────────────────────────────────┤
│  Data (lake, warehouse, pipelines, labeling, governance)      │
├─────────────────────────────────────────────────────────────┤
│  Platform (K8s, GPUs, IAM, observability, cost, security)   │
└─────────────────────────────────────────────────────────────┘

Layer by layer

1. Data — raw events, documents, images; cleaned, labeled, and governed. Bad data caps every model.
2. Training (optional for many app devs) — notebooks or pipelines produce a model artifact; often you consume a vendor or open model instead.
3. Model serving — HTTP/gRPC endpoint, batch jobs, or edge runtime; watch latency, throughput, and cost per token.
4. Retrieval & memory — chunk documents, embed, store in Pinecone, pgvector, OpenSearch, etc.; retrieve top-k for RAG.
5. Application logic — auth, rate limits, prompt templates, tool definitions, guardrails, fallbacks.
6. Observability — traces, prompt/response logging (with PII redaction), eval dashboards, drift alerts.

If you come from platform engineering, this should feel familiar: it is microservices plus data plus GPUs plus new failure modes (nondeterminism, prompt injection, runaway token cost).

Foundations: what to learn (and in what order)

1. Software engineering (non-negotiable)

• One language deeply: Python is the default for AI; TypeScript or Go matter for product APIs and platform.
• Git, testing, REST/gRPC, containers, basic SQL, and debugging production issues.
• Security basics: secrets management, least-privilege IAM, input validation (including against prompt injection).

2. Math and statistics (enough, not everything)

• Linear algebra (vectors, matrices), calculus intuition (gradients), probability, and descriptive statistics.
• For application-focused AI devs, depth in evaluation design often beats manual backprop derivations.

3. Classical ML concepts

• Supervised vs unsupervised learning, train/validation/test splits, overfitting, regularization.
• Common algorithms: linear/logistic regression, trees, random forests, basic neural nets.
• When not to use ML—a rules engine or SQL may be cheaper and clearer.

4. Deep learning and transformers (conceptual + one hands-on path)

• Neural networks, CNNs (vision), RNNs (legacy sequence), transformers and attention.
• Pre-training vs fine-tuning vs prompting; parameter count vs capability vs cost.

5. Generative AI and product patterns

• Prompt design, structured outputs (JSON schema), streaming, tool use, multi-step agents.
• RAG end-to-end: chunking strategy, embeddings, retrieval quality, citation in UI.

Tech stack: languages, frameworks, models, cloud

Languages

• Python — PyTorch, Hugging Face, LangChain, notebooks, training scripts.
• TypeScript / JavaScript — full-stack AI apps, Vercel AI SDK, edge deployments.
• SQL — features, analytics, and vector extensions (e.g. pgvector).

ML & deep learning frameworks

• PyTorch — research and production training; huge ecosystem.
• TensorFlow / Keras — still common in enterprise and mobile (TF Lite).
• JAX — Google stack, high-performance research.
• scikit-learn — classical ML baselines and pipelines.
• Hugging Face Transformers — download, fine-tune, and serve thousands of open models.

Application & orchestration

• LangChain, LlamaIndex, Semantic Kernel — chains, agents, connectors (learn concepts; avoid framework magic without understanding).
• OpenAI / Anthropic / Google APIs — managed frontier models.
• Ollama, llama.cpp, vLLM — local or self-hosted open models.

Models (families, not an exhaustive catalog)

Family	Examples	Typical use
Closed API LLMs	GPT-4o, Claude, Gemini	General reasoning, agents, codegen
Open weights LLMs	Llama, Mistral, Qwen	Self-host, fine-tune, air-gapped
Code models	Codex-style, Code Llama, StarCoder	IDE assistants, PR review
Embedding models	text-embedding-3, BGE, E5	RAG, search, clustering
Vision / multimodal	GPT-4V-class, LLaVA	Docs, UI, image Q&A
Speech	Whisper	Transcription, voice apps

Choose models by task fit, latency, cost, privacy, and eval scores—not leaderboard hype alone.

Data & vector stores

• Warehouses: Snowflake, BigQuery, Redshift; lakes on S3 + Spark/DuckDB.
• Vectors: Pinecone, Weaviate, Chroma, pgvector, OpenSearch k-NN.
• Pipelines: Airflow, Dagster, dbt—for reliable features and document ingestion.

Cloud & MLOps (especially on AWS)

• Amazon SageMaker — training, endpoints, pipelines, model registry.
• Bedrock — managed foundation models and knowledge bases.
• Storage/compute: S3, ECR, ECS/EKS with GPU node pools, Lambda for light inference.
• Observability: CloudWatch, Prometheus, LangSmith, Arize, Weights & Biases.

My notes from structured learning: Machine Learning Foundations, Generative AI Foundations, and Data Engineering.

How to develop: a practical learning path

Phase 0 — Baseline (2–4 weeks)

• Solidify Python or your main language, Git, and one small API project.
• Complete an intro ML course (Andrew Ng’s ML specialization or equivalent).

Phase 1 — Build one classical ML project (3–6 weeks)

• Pick a tabular dataset (Kaggle or public gov data); train a baseline, measure metrics, deploy a simple Flask/FastAPI predict endpoint.

Phase 2 — Generative AI application (4–8 weeks)

• Build a RAG chatbot over your own docs: ingest PDFs/Markdown, chunk, embed, retrieve, answer with citations.
• Add evals: 20–50 golden questions; track answer quality over prompt/model changes.

Phase 3 — Production habits (ongoing)

• CI for prompts and schemas, cost dashboards, red-team basic prompt injection cases.
• Read ISO/IEC 42001 themes if you touch regulated or enterprise AI.

Portfolio tip: One repo with README architecture diagram, sample eval results, and clear “what I’d do next in production” beats five half-finished tutorials.

Common pitfalls

• Skipping software craft — notebooks do not replace tests, APIs, or observability.
• Tool-chasing — new frameworks weekly; master patterns (RAG, eval, serving) first.
• No evals — demos lie; metrics and human review decide if you ship.
• Ignoring cost and latency — token bills and p95 latency kill products silently.
• Trusting the model for facts — always ground or verify high-stakes answers.

Where the field is going

• Smaller, cheaper models for edge and high-volume tasks; frontier models for hard reasoning.
• Multimodal products — text, image, audio, and video in one workflow.
• Agents with guardrails — tool use plus policy, audit logs, and human approval steps; portable tool wiring via MCP servers in depth.
• Regulation and standards — AI management systems, EU AI Act-style risk tiers, sector rules.
• AI-native platforms — inference, vectors, and evals as first-class cloud primitives (see also cloud platform evolution).

The developers who thrive will combine product sense, solid engineering, and honest uncertainty about model limits—with the habit of measuring what actually helps users.

Further reading

• Historical paradigms of AI — how the field evolved.
• fast.ai, Hugging Face course, DeepLearning.AI short courses — hands-on depth.
• Papers: “Attention Is All You Need”; OpenAI / Anthropic system cards for capability and safety framing.
• Books: Designing Machine Learning Systems (Chip Huyen); Build a Large Language Model (From Scratch) (Sebastian Raschka) for implementers.

Blog index · RAG in depth · Foundation models · AI history · Generative AI Foundations · ISO 42001 AI management

Back to blog list