ishavasya

10. API Design Fundamentals

19 min read

What this note is really about

When people first learn API design, they often focus on surface details such as:

  • routes
  • JSON shapes
  • HTTP verbs
  • status codes
  • framework annotations

Those matter, but they are not the foundation.

The foundation is this:

an API is a contract for interaction between software components

That contract must be:

  • understandable
  • predictable
  • evolvable
  • safe
  • efficient enough for its purpose

This note is framework-agnostic on purpose.

It is not about one web framework, one RPC system, or one serialization library.

It is about the fundamental thinking that should remain valid across:

  • REST APIs
  • RPC APIs
  • internal service APIs
  • library APIs
  • SDK APIs
  • event-driven interfaces

The goal is to become clear-headed about API design at a conceptual level, so implementation choices become easier and more coherent.

Start with the broadest definition

An API is an application programming interface.

That means it is a defined way for one software component to interact with another.

That component may be:

  • another process
  • another service
  • a library in the same application
  • a browser client talking to a server
  • a mobile app talking to a backend
  • an internal module exposing capabilities to the rest of the codebase

So an API is not inherently:

  • HTTP
  • JSON
  • REST
  • public internet

Those are common forms.

But the real idea is:

an API exposes capabilities while hiding implementation details

That already tells you something important:

good API design is not only a networking problem.

It is an abstraction problem.

Why APIs exist

APIs exist because software systems need boundaries.

Without boundaries:

  • everything knows about everything
  • changes ripple everywhere
  • responsibilities become unclear
  • reuse becomes hard
  • independent evolution becomes difficult

An API gives one part of the system a stable way to ask another part to do something or provide information.

That means an API is about:

  • communication
  • separation of concerns
  • controlled access
  • long-term maintainability

So when designing an API, you are not merely designing endpoints.

You are designing a boundary between systems or subsystems.

The first principle: design from use cases, not from framework mechanics

Bad API design often starts like this:

  • what routes should I create?
  • what controller should I add?
  • what verbs does this framework prefer?

That is backward.

The better starting point is:

  • who is the consumer?
  • what do they need to accomplish?
  • what information do they need?
  • what actions do they need to perform?
  • what constraints and failure cases exist?

So API design should begin from:

  • domain understanding
  • user or client workflows
  • data ownership
  • business rules

Framework mechanics come later.

This one habit separates strong API thinking from shallow endpoint design.

The second principle: design the contract, not just the transport

A transport is how bytes move:

  • HTTP
  • gRPC
  • message broker
  • WebSocket

The contract is what the interaction means:

  • operation names
  • resource structure
  • request shape
  • response shape
  • error behavior
  • guarantees
  • ordering expectations
  • versioning promises

Transport matters, but contract matters more.

A technically working HTTP endpoint can still be a poor API if:

  • its behavior is unclear
  • naming is confusing
  • errors are inconsistent
  • clients cannot rely on stable semantics

So the heart of API design is semantic clarity.

The third principle: the consumer experience matters

An API is used by humans through code.

That means the API must support:

  • discoverability
  • predictability
  • ease of correct usage
  • difficulty of incorrect usage

A strong API reduces ambiguity.

A weak API forces every consumer to guess:

  • what a field means
  • whether an operation is safe to retry
  • which fields are optional
  • which errors are expected
  • how pagination works
  • what changes are backward compatible

So API design is partly about developer ergonomics.

Not in a superficial way, but in a correctness and maintenance way.

Internal APIs and external APIs

Not all APIs have the same audience.

Internal APIs

Used within one organization or one codebase ecosystem.

These can sometimes move faster, but they still need discipline because internal chaos becomes system-wide chaos.

Partner or public APIs

Used by external consumers you do not control.

These require even more:

  • stability
  • documentation
  • compatibility discipline
  • operational reliability
  • explicit versioning and deprecation strategy

The more external the audience, the stronger the API contract must be.

Synchronous APIs and asynchronous APIs

Another important distinction:

Synchronous request-response API

Caller sends request and waits for result.

Examples:

  • HTTP request returning JSON
  • local method call
  • RPC request

Asynchronous API

Caller initiates work, and completion happens later.

Examples:

  • event-driven messaging
  • job submission with later polling or callback
  • webhook-based completion notification

This distinction matters because API design changes when the operation is:

  • fast and immediate
  • long-running
  • potentially retried
  • eventually consistent

Do not force long-running workflows into fake immediate APIs without thinking through the consequences.

Start from the domain

A strong API usually reflects the actual problem domain rather than raw storage structure.

Bad API:

  • mirrors database tables directly
  • exposes implementation internals
  • uses field names that only make sense inside the current codebase

Better API:

  • exposes concepts meaningful to consumers
  • groups behavior around domain operations
  • models entities and actions the way the business actually thinks about them

Example:

Instead of exposing:

  • tbl_order_hdr
  • cust_fk
  • ord_stat_cd

design around:

  • orders
  • customerId
  • status

This sounds obvious, but many APIs fail here.

API design should be shaped by domain meaning first, not by storage trivia.

Resource-oriented and operation-oriented design

Many APIs lean toward one of two styles.

Resource-oriented design

The API revolves around identifiable things such as:

  • users
  • orders
  • products
  • invoices

Operations are expressed in relation to those things.

This style fits naturally with many HTTP APIs.

Operation-oriented design

The API revolves around explicit actions such as:

  • calculate quote
  • submit payment
  • reconcile account
  • generate report

This style is common in RPC systems and also appears in HTTP APIs where behavior matters more than CRUD.

Neither style is universally correct.

The right choice depends on the domain.

Many real APIs mix both:

  • resource-oriented structure for core entities
  • operation endpoints for important workflows or commands

The real question is not “REST or not?”

It is:

“What model makes the contract most truthful and clear?”

CRUD is not the whole story

Many beginners design APIs as if every problem is:

  • create
  • read
  • update
  • delete

That works for simple administrative systems.

But many real domains are richer.

Examples:

  • approve invoice
  • cancel order
  • capture payment
  • publish article
  • resend verification email
  • retry job

These are not well-described as generic CRUD alone.

So do not let CRUD thinking flatten business meaning.

If an action matters in the domain, model it explicitly.

Naming is foundational

Naming quality strongly shapes API quality.

Good names are:

  • clear
  • stable
  • domain-meaningful
  • unsurprising

Bad names are:

  • vague
  • overly abbreviated
  • implementation-specific
  • inconsistent across similar concepts

Examples of better naming:

  • orderId instead of oid
  • createdAt instead of crt_ts
  • isArchived instead of flag1

Good naming matters because the API contract gets read and reused repeatedly.

Names are not cosmetic.

They are part of the interface semantics.

Consistency is a major design principle

Consistency is one of the highest-leverage qualities in an API.

If one part of the API uses:

  • one naming style
  • one pagination style
  • one date format
  • one error structure

but another part invents its own approach, the API becomes harder to use than necessary.

Consistency should apply across:

  • naming
  • field casing
  • identifier patterns
  • filtering rules
  • sorting conventions
  • status representation
  • error format
  • authentication behavior
  • retry semantics

An API that is not perfectly elegant but is highly consistent is often more usable than one with many clever but inconsistent choices.

Inputs and outputs are not just data bags

Requests and responses should be shaped intentionally.

Ask:

  • what is required?
  • what is optional?
  • what is server-controlled?
  • what is computed?
  • what belongs in this operation versus another one?

Weak design often exposes raw object graphs or database entities without carefully deciding what the contract should be.

Strong design makes clear:

  • which fields the client may send
  • which fields are returned
  • which fields are immutable
  • which fields are derived
  • which fields are for identity, state, metadata, or relationships

This keeps the contract focused and easier to evolve safely.

Data types and representation choices

An API is not only about what fields exist.

It is also about how values are represented.

Important examples:

  • strings vs numbers
  • booleans
  • enums or status strings
  • timestamps
  • currency
  • identifiers
  • nullability
  • arrays vs sets vs maps

These choices matter more than they first appear.

For example:

  • timestamps need clear timezone and format conventions
  • money should not be represented casually as binary floating-point if precision matters
  • status values should be stable and documented

Value representation mistakes can create long-term pain.

Explicitness beats hidden assumptions

Consumers should not have to infer critical rules from trial and error.

Be explicit about:

  • required fields
  • units
  • currencies
  • timezone handling
  • ordering guarantees
  • default behaviors
  • maximum limits
  • idempotency expectations
  • retry safety

If the API assumes something important, say it in the contract and documentation.

Hidden assumptions become integration bugs.

Validation is part of the contract

Every API has rules about valid input.

Examples:

  • required fields
  • field length limits
  • format constraints
  • allowed state transitions
  • cross-field consistency

Validation is not only an implementation detail.

It is part of what the API means.

A well-designed API makes validation behavior:

  • predictable
  • consistent
  • specific enough to act on

Good validation errors help clients fix their requests.

Bad validation errors force guesswork.

Error design is first-class API design

Many APIs are designed carefully for success and sloppily for failure.

That is a mistake.

Consumers need to know:

  • what can fail
  • why it failed
  • whether retry is reasonable
  • whether the request was invalid, unauthorized, conflicting, or temporarily impossible

A strong error model usually includes:

  • stable error codes or categories
  • human-readable message
  • machine-readable details where useful
  • consistent structure
  • trace or correlation identifiers when operationally helpful

The exact format can vary, but error behavior must be coherent.

Error handling is not a side topic.

It is part of the API contract.

Idempotency

This is one of the most important API design concepts.

An operation is idempotent if performing it multiple times with the same intended effect does not keep changing the result.

Examples:

  • setting a status to cancelled may be idempotent if repeating it changes nothing after the first success
  • charging a payment is often not naturally idempotent unless the API provides a specific idempotency mechanism

Why does this matter?

Because retries happen in real systems:

  • network failures
  • timeouts
  • proxy retries
  • client uncertainty about whether the first request succeeded

If the caller retries, what happens?

That question is critical.

For operations with side effects, APIs often need explicit idempotency strategies such as:

  • idempotency keys
  • operation tokens
  • conflict detection

If you ignore this, clients can easily create duplicates or inconsistent side effects.

Safety and side effects

Clients need to understand whether an operation:

  • only reads
  • mutates state
  • triggers external actions
  • may have irreversible effects

This matters for:

  • retry behavior
  • caching
  • user confirmation
  • operational auditing

A clean API design makes side effects clear.

If an operation sends emails, charges money, publishes content, or starts background jobs, that should not be hidden behind a misleadingly harmless-looking call.

Partial updates and replacement

One common design problem is how to update data.

Important questions:

  • is the client replacing the full resource?
  • only updating selected fields?
  • issuing a domain command rather than a structural edit?

These are not equivalent.

If your update model is unclear, clients may:

  • accidentally erase fields
  • overwrite concurrent changes
  • misunderstand defaults

So update semantics must be clear:

  • replace
  • patch
  • command-based action

Do not blur these carelessly.

Concurrency control and conflicting updates

Real systems often have multiple clients modifying the same data.

Questions arise:

  • what if two clients update the same record at once?
  • what if a client edits stale data?
  • what if order state changes between read and write?

API design should think about concurrency at the contract level, not only inside the database.

Important strategies include:

  • version fields
  • ETags
  • optimistic concurrency checks
  • conflict responses
  • explicit state transition rules

Without this, updates can silently overwrite each other.

That is often a serious correctness problem.

Pagination

Large collections cannot usually be returned all at once forever.

So collection APIs need pagination.

Important design questions:

  • page number or cursor?
  • how does sorting interact with pagination?
  • what happens when underlying data changes during traversal?
  • are next-page links or tokens provided?
  • what are the max page sizes?

Offset-based pagination is simple but can become unstable or inefficient on changing large datasets.

Cursor-based pagination is often more reliable for large or frequently changing collections.

Pagination is not just a performance tweak.

It is part of the contract for how clients navigate data.

Filtering, sorting, and searching

Collection endpoints often need query capabilities.

Important concerns:

  • field naming consistency
  • allowed operators
  • case sensitivity rules
  • exact match vs partial match
  • sort direction conventions
  • default ordering

Do not make consumers guess:

  • whether status=active is exact
  • whether string filtering is case-sensitive
  • whether multiple filters combine with AND or OR

Query semantics should be explicit and predictable.

Response size and field selection

APIs often fail by returning either:

  • too little information, forcing many extra calls
  • too much information, wasting bandwidth and coupling clients to unnecessary data

Good design asks:

  • what does the common consumer actually need?
  • should related summaries be embedded?
  • should heavy expansions be optional?
  • should field selection exist?

This is partly about performance, but also about contract clarity.

Returning the whole internal model “just in case” is usually weak design.

Performance is part of design, but not the only goal

An API should be efficient enough for its purpose.

That includes thinking about:

  • payload size
  • round trips
  • batching
  • pagination
  • caching
  • server cost
  • latency

But performance should not be optimized blindly at the expense of clarity.

A good API balances:

  • usability
  • correctness
  • evolvability
  • performance

Performance concerns should shape the contract where needed, but not turn it into something unreadable.

Caching

Some API responses are expensive to compute or slow to fetch repeatedly.

Caching can help, but only when semantics are clear.

Important questions:

  • is the response cacheable?
  • for how long?
  • what invalidates it?
  • is the data user-specific?
  • is stale data acceptable?

Caching works best when the API clearly distinguishes:

  • stable reads
  • highly dynamic reads
  • user-specific responses
  • mutation-triggering operations

Good caching design is easier when the API semantics are already clean.

Versioning

APIs change.

That is unavoidable.

The real question is how change is managed.

Important concerns:

  • what counts as a breaking change?
  • how are additions handled?
  • how long are old versions supported?
  • how are deprecations communicated?
  • what migration path exists?

Breaking changes often include:

  • removing fields
  • renaming fields
  • changing requiredness
  • changing semantics
  • changing error behavior
  • changing ordering or pagination guarantees

Adding new optional fields is often backward compatible, but even that depends on clients not making bad assumptions.

Versioning is not just about path names like /v1/.

It is about compatibility discipline.

Backward compatibility

A strong API treats compatibility as a serious design constraint.

Clients may:

  • upgrade slowly
  • cache assumptions
  • depend on undocumented behavior
  • be maintained by other teams

So changes should be evaluated from the client perspective.

Ask:

  • will existing clients still work?
  • will they misinterpret the response?
  • will parsing fail?
  • will retries behave differently?

Compatibility is partly technical and partly behavioral.

Security is not an add-on

Every API boundary is also a security boundary.

Important concerns include:

  • authentication
  • authorization
  • least privilege
  • input validation
  • secret handling
  • transport security
  • rate limiting
  • abuse prevention
  • auditability

An API should make it clear:

  • who is calling
  • what they are allowed to do
  • how sensitive operations are protected

Weak security design can make even a functionally correct API dangerous.

Authentication vs authorization

These are often confused.

Authentication

Who is the caller?

Authorization

What is the caller allowed to do?

An API can authenticate a user correctly and still have poor authorization design.

Authorization must be thought through at the level of:

  • operations
  • resources
  • tenancy boundaries
  • data visibility
  • admin versus normal behavior

Rate limiting and quotas

Real APIs need protection against overload and abuse.

Important mechanisms include:

  • rate limiting
  • quotas
  • concurrency limits
  • request size limits

These are not only operational concerns.

They influence how consumers should design their integrations:

  • how aggressively to retry
  • how to batch work
  • how to back off under pressure

So if limits exist, they should be made explicit.

Long-running operations

Some operations cannot complete immediately.

Examples:

  • export report
  • video transcoding
  • bulk import
  • document analysis
  • workflow execution

Do not pretend these are instant if they are not.

Common strategies include:

  • submit job and return job identifier
  • allow polling for status
  • send webhook or callback on completion
  • expose operation result later

Long-running workflows need API contracts that acknowledge time and state transitions explicitly.

Async workflows and eventual consistency

Sometimes an API call returns before all downstream effects are complete.

Examples:

  • order accepted, fulfillment starts later
  • message accepted, processing happens asynchronously
  • data write replicated eventually

This is fine if the contract is honest.

Clients need to know:

  • what is guaranteed immediately
  • what is pending
  • how to observe completion or failure
  • whether reads are instantly up to date

If the system is eventually consistent, say so clearly where it matters.

Webhooks and event-driven interfaces

Not all APIs are request-response only.

Some systems also expose:

  • webhooks
  • event streams
  • message-based integration

These raise additional design concerns:

  • delivery guarantees
  • retry behavior
  • ordering
  • deduplication
  • signature verification
  • idempotent consumers

Event-driven APIs are still APIs.

They simply express interaction through messages over time rather than immediate response pairs.

Observability and supportability

An API is easier to operate when requests can be traced and failures investigated.

Important concerns include:

  • correlation IDs
  • request IDs
  • stable error codes
  • useful logs
  • metrics
  • audit trails where necessary

This is not only an ops detail.

A well-supported API helps consumers diagnose issues without blind guessing.

Documentation is part of the API

An undocumented API is not truly well-designed, even if the endpoint behavior is technically good.

Consumers need to know:

  • available operations
  • request and response formats
  • authentication rules
  • limits
  • error behavior
  • examples
  • versioning policy
  • retries and idempotency expectations

Documentation is not a separate afterthought.

It is part of the usability of the contract itself.

Examples matter

Examples help clarify intent far better than abstract field lists alone.

Good examples show:

  • typical requests
  • typical responses
  • validation errors
  • pagination flow
  • async job flow
  • retry-safe usage

Examples do not replace formal specification, but they dramatically improve understanding.

Framework independence and transport independence

A strong API design should survive translation across tools.

If your API only makes sense because one framework generated a certain shape automatically, that is a warning sign.

The core questions should remain stable across technology choices:

  • what capability is exposed?
  • what are the inputs and outputs?
  • what are the guarantees?
  • what are the failure modes?
  • what evolves safely?

Transport-specific details matter, but they should serve the contract, not define it thoughtlessly.

Common weak patterns

Weak API design often shows one or more of these problems:

  • table-shaped endpoints with no domain thinking
  • inconsistent naming and error behavior
  • unclear update semantics
  • missing idempotency strategy
  • no pagination discipline
  • overexposed internal data models
  • poor versioning discipline
  • vague documentation
  • hidden side effects
  • security added as an afterthought

These are not cosmetic flaws.

They create long-term integration cost and operational pain.

Common strong patterns

Strong API design often shows these qualities:

  • domain-driven naming
  • consistent conventions
  • explicit contract semantics
  • clear distinction between reads, updates, and commands
  • good validation and error structure
  • safe retry and idempotency design
  • scalability-aware collection handling
  • honest modeling of long-running work
  • compatibility discipline
  • strong security and documentation

This is what “mature API design” tends to look like in practice.

A practical API design workflow

When designing an API from scratch, a useful sequence is:

  1. Understand the domain and consumer workflows.
  2. Identify the capabilities the API must expose.
  3. Decide whether each capability is best modeled as a resource interaction, an explicit operation, an event, or a long-running job.
  4. Define inputs, outputs, identifiers, and state transitions clearly.
  5. Design validation, errors, retry behavior, and idempotency intentionally.
  6. Consider pagination, filtering, performance, and payload shape.
  7. Define authentication, authorization, and abuse controls.
  8. Plan compatibility, versioning, and deprecation from the beginning.
  9. Document with examples and edge cases.
  10. Review the contract from the consumer’s point of view, not only the server implementer’s point of view.

That workflow is far more valuable than memorizing endpoint templates.

The most important conceptual thread

If you want one flow that ties the whole topic together, it is this:

An API is a boundary contract between software components. Good API design starts from the domain and consumer use cases, not from framework-generated routes or storage tables. The designer must choose meaningful capabilities, shape requests and responses intentionally, define names and data representations clearly, and make validation, error behavior, retries, side effects, idempotency, pagination, and security explicit. Because APIs live over time, they must also be designed for compatibility, documentation, observability, and evolution. The transport may be HTTP, RPC, messaging, or something else, but the fundamental challenge is always the same: expose useful capability through a predictable, stable, comprehensible contract.

What you should come away knowing

You should leave this topic with these ideas clearly separated:

  1. An API is a contract, not just an endpoint.
  2. API design starts from use cases and domain meaning.
  3. Resource structure and operation design are both valid tools.
  4. Naming, consistency, and representation choices are foundational.
  5. Validation and errors are part of the interface, not implementation leftovers.
  6. Idempotency, retries, and side effects must be designed intentionally.
  7. Pagination, filtering, and response shaping are part of usability and scalability.
  8. Security, rate limits, and long-running workflow handling are core concerns.
  9. Versioning is really about compatibility discipline over time.
  10. Documentation and observability are part of the API experience.

Bottom line

API design is the discipline of creating a clear, stable, and usable contract between software components. The deepest issues are not framework syntax or route decoration, but semantic ones: what capabilities are exposed, how they are named, what guarantees they provide, how failures behave, how clients evolve safely, and how the boundary remains understandable over time. Once those fundamentals are clear, the same thinking transfers across frameworks, protocols, and languages.

Graph View