Skip to Content

Preface

Agent Workflow Problem

Agent Workflow Example

In the traditional agent development method, whenever the agent’s functionality is expanded, AI developers had drawn more and more complex agent workflows and graphs. However, no matter how complex the workflow graph is drawn, the accuracy of the agent has been significantly reduced as the functionality has expanded.

It’s because whenever a new node is added to the agent graph, the number of processes to go through increases, and the success rate decreases as a Cartesian Product by the number of processes. For example, if five agent nodes are sequentially listed up, and each node has a 80 % success rate, the final success rate becomes 32.77 % (0.85).

To hedge the Cartesian Product of success rate, AI developers need to construct a much more complex graph to independently partition each event. This inevitably makes AI agent development difficult and makes it difficult to respond to changing requirements, such as adding or modifying new features.

Document Driven Development

Here I propose a new paradigm; “Document Driven Development”.

Take every responsibilities to the LLM (Large Language Model) function calling. And you just concentrate on documentation (comment) for each function independently. Then the success rate would be independent for each function, so that the cartesian product to success rate would not be applied. Also, since you only need to adjust the list of functions to be called, you can develop more flexibly and quickly than any other methodology.

I call this new paradigm as “Document Driven Development”, and recommend to combine with the Compiler Driven Development. Just believe @agentica, and give up drawing complex agent graphs (workflows). You just concentrate on documentation for each function. @agentica and LLM function calling will fully take care of the rest.

Demonstration of Efficiency


Shopping backend server project for enterprise level chatbot demonstration.

@samchon/shopping-backend is a shopping mall backend server project consisting of a total of 289 API functions which can cover most features of e-commerce. It supports authentication, product managenent, order purchasing, delivery tracking, discount coupon, deposit and withdrawal, and so on.

@samchon/shopping-backend is a typical project that can never be achieved with the traditional agent development way utilizing agent workflow graphs. Will you draw infinite number of agent workflows, or just document each function? Will you re-draw the infinite number of agent workflows whenever a new feature is added, or just add a new function with documentation comment?

@samchon/shopping-backend shows how “Document Driven Development” is effective in the AI agent development. Only the Compiler Driven Development and “Document Driven Development” methodologies can accomplish such enterprise level AI agent development. Make it easily and safely with @agentica.

Functional Documentation

Description Comment

example/src/BbsArticleService.ts
import { tags } from "typia"; import { v4 } from "uuid"; import { IBbsArticle } from "./IBbsArticle"; export class BbsArticleService { private readonly articles: IBbsArticle[] = []; /** * Get all articles. * * List up every articles archived in the BBS DB. * * @returns List of every articles */ public index(): IBbsArticle[] { return this.articles; } /** * Create a new article. * * Writes a new article and archives it into the DB. * * @param props Properties of create function * @returns Newly created article */ public create(props: { /** * Information of the article to create */ input: IBbsArticle.ICreate; }): IBbsArticle { const article: IBbsArticle = { id: v4(), title: props.input.title, body: props.input.body, thumbnail: props.input.thumbnail, created_at: new Date().toISOString(), updated_at: new Date().toISOString(), }; this.articles.push(article); return article; } /** * Update an article. * * Updates an article with new content. * * @param props Properties of update function * @param input New content to update */ public update(props: { /** * Target article's {@link IBbsArticle.id}. */ id: string & tags.Format<"uuid">; /** * New content to update. */ input: IBbsArticle.IUpdate; }): void { const article: IBbsArticle | undefined = this.articles.find( (a) => a.id === props.id, ); if (article === undefined) throw new Error("Unable to find the matched article."); if (props.input.title !== undefined) article.title = props.input.title; if (props.input.body !== undefined) article.body = props.input.body; if (props.input.thumbnail !== undefined) article.thumbnail = props.input.thumbnail; article.updated_at = new Date().toISOString(); } /** * Erase an article. * * Erases an article from the DB. * * @param props Properties of erase function */ public erase(props: { /** * Target article's {@link IBbsArticle.id}. */ id: string & tags.Format<"uuid">; }): void { const index: number = this.articles.findIndex((a) => a.id === props.id); if (index === -1) throw new Error("Unable to find the matched article."); this.articles.splice(index, 1); } }

Describe for proper function selecting.

When writing description comment on each function, describe only about purpose of the function, concentrating on how to induce the agent to properly select the function during the conversation. Don’t describe about its DTO like parameters or return types.

It’s because @agentica has separated the candidate function selecting process and arguments filling of the selected function process independently in its internal agents’ orchestration. Also, description of the function is restricted to maximum 1,023 characters in the LLM providers like OpenAI. Therefore, no need to describe abouot DTO in the description comment written on the function.

Dependency Relationship

@samchon/shopping-backend/ShoppingSaleController
import { TypedBody, TypedRoute, TypedParam } from "@nestia/core"; import { Controller } from "@nestjs/common"; import { tags } from "typia"; import { IPage } from "@samchon/shopping-api/lib/structures/common/IPage"; import { IShoppingActorEntity } from "@samchon/shopping-api/lib/structures/shoppings/actors/IShoppingActorEntity"; import { IShoppingSale } from "@samchon/shopping-api/lib/structures/shoppings/sales/IShoppingSale"; import { IShoppingControllerProps } from "../IShoppingControllerProps"; export function ShoppingSaleController<Actor extends IShoppingActorEntity>( props: IShoppingControllerProps, ) { @Controller(`shoppings/${props.path}/sales`) abstract class ShoppingSaleController { /** * List up every summarized sales. * * List up every {@link IShoppingSale.ISummary summarized sales}. * * As you can see, returned sales are summarized, not detailed. It does not * contain the SKU (Stock Keeping Unit) information represented by the * {@link IShoppingSaleUnitOption} and {@link IShoppingSaleUnitStock} types. * If you want to get such detailed information of a sale, use * `GET /shoppings/customers/sales/{id}` operation for each sale. * * > If you're an A.I. chatbot, and the user wants to buy or compose * > {@link IShoppingCartCommodity shopping cart} from a sale, please * > call the `GET /shoppings/customers/sales/{id}` operation at least once * > to the target sale to get detailed SKU information about the sale. * > It needs to be run at least once for the next steps. * * @param input Request info of pagination, searching and sorting * @returns Paginated sales with summarized information * @tag Sale * * @author Samchon */ @TypedRoute.Patch() public index( @props.AuthGuard() actor: Actor, @TypedBody() input: IShoppingSale.IRequest, ): Promise<IPage<IShoppingSale.ISummary>>; /** * Get a sale with detailed information. * * Get a {@link IShoppingSale sale} with detailed information including * the SKU (Stock Keeping Unit) information represented by the * {@link IShoppingSaleUnitOption} and {@link IShoppingSaleUnitStock} types. * * > If you're an A.I. chatbot, and the user wants to buy or compose a * > {@link IShoppingCartCommodity shopping cart} from a sale, please call * > this operation at least once to the target sale to get detailed SKU * > information about the sale. * > * > It needs to be run at least once for the next steps. In other words, * > if you A.I. agent has called this operation to a specific sale, you * > don't need to call this operation again for the same sale. * > * > Additionally, please do not summarize the SKU information. Just show * > the every options and stocks in the sale with detailed information. * * @param id Target sale's {@link IShoppingSale.id} * @returns Detailed sale information * @tag Sale * * @author Samchon */ @TypedRoute.Get(":id") public at( @props.AuthGuard() actor: Actor, @TypedParam("id") id: string & tags.Format<"uuid">, ): Promise<IShoppingSale>; } return ShoppingSaleController; }

Dependency relationship, also writes in the description comment.

If there’re some functions that should be called before or after the function, describe the dependency relationship in the description comment. If the LLM function calling schema comes from a TypeScript class, then refer the target function name. Otherwise the function comes from the backend server, refer the target function’s method and name.

For example, in the demonstration project @samchon/shopping-backend, there are two functions shoppings.sales.index and shoppings.sales.at. The first function lists up all sales in the market with summarized information, and the seconnd function shows the detailed information of a specific sale. To describe such dependency relationship, ShoppingSaleController’s description comment is written as above.

Selection Benchmark

Benchmark your documentation quality.

When you’ve completed documentation for all functions, benchmark your documentation quality by function select benchmark of @agentica/benchmark module. Rather than testing the documentation quality by yourself by running AI chatbot manually, making benchmark program is much more effective.

In the @samchon/shopping-backend project, it has measured documentation quality by below user prompt. And in the benchmark, success rate of function selecting was always 100 %.

I wanna see every sales in the shopping mall

And then show me the detailed information about the Macbook.

After that, select the most expensive stock from the Macbook, and put it into my shopping cart. And take the shopping cart to the order.

At last, I’ll publish it by cash payment, and my address is

  • country: South Korea
  • city/province: Seoul
  • department: Wrtn Apartment
  • Possession: 101-1411

Schema Documentation

DTO Description

undefined

@samchon/shopping-backend/IShoppingSaleUnitStock
/** * Final component information on units for sale. * * `IShoppingSaleUnitStock` is a subsidiary entity of {@link IShoppingSaleUnit} * that represents a product catalog for sale, and is a kind of final stock that is * constructed by selecting all {@link IShoppingSaleUnitSelectableOption options} * (variable "select" type) and their * {@link IShoppingSaleUnitOptionCandidate candidate} values in the belonging unit. * It is the "good" itself that customers actually purchase. * * - Product Name) MacBook * - Options * - CPU: { i3, i5, i7, i9 } * - RAM: { 8GB, 16GB, 32GB, 64GB, 96GB } * - SSD: { 256GB, 512GB, 1TB } * - Number of final stocks: 4 * 5 * 3 = 60 * * For reference, the total number of `IShoppingSaleUnitStock` records in an * attribution unit can be obtained using Cartesian Product. In other words, the * value obtained by multiplying all the candidate values that each * (variable "select" type) option can have by the number of cases is the total * number of final stocks in the unit. * * Of course, without a single variable "select" type option, the final stocks * count in the unit is only 1. * * @author Samchon */ export interface IShoppingSaleUnitStock { ... }

Describe within framework of entity.

When documenting DTO (Data Transfer Object) types, it would better to describe them within framework of entity. Try to describe conceptually what each DTO type represents, and explain what the related types (entities) are. When explaining the related types, using {@link Target} tag is recommended.

In actually, above DTO types are come from @samchon/shopping-backend, and their descriptions are similar with ERD (Entity Relationship Diagram) manual  explaining entities. If you open the ERD manual and read the description of shopping_sale_snapshot_unit_stocks entity, you may easily understand what I am talking about.

Also, if concept of a DTO type is difficult to understand for human, it is difficult for AI agent too. In that case, it would better to use some examples or analogies. For example, above IShoppingSaleUnitStock contains SKU (Stock Keeping Unit) concept, and it is hard to understand for someone who is not an e-commerce professional. So, IShoppingSaleUnitStock is explaining the SKU concept with an example of MacBook’s (CPU, RAM, and SSD) options.

Property Description

undefined

@samchon/shopping-backend/IShoppingCouponRestriction
import { tags } from "typia"; /** * Restriction information of the coupon. * * @author Samchon */ export interface IShoppingCouponRestriction { /** * Access level of coupon. * * - public: possible to find from public API * - private: unable to find from public API * - arbitrarily assigned by the seller or administrator * - issued from one-time link */ access: "public" | "private"; /** * Exclusivity or not. * * An exclusive discount coupon refers to a discount coupon that has an * exclusive relationship with other discount coupons and can only be * used alone. That is, when an exclusive discount coupon is used, no * other discount coupon can be used for the same * {@link IShoppingOrder order} or {@link IShoppingOrderGood good}. * * Please note that this exclusive attribute is a very different concept * from multiplicative, which means whether the same coupon can be * multiplied and applied to multiple coupons of the same order, so * please do not confuse them. */ exclusive: boolean; /** * Limited quantity issued. * * If there is a limit to the quantity issued, it becomes impossible * to issue tickets exceeding this value. * * In other words, the concept of N coupons being issued on * a first-come, first-served basis is created. */ volume: null | (number & tags.Type<"uint32">); /** * Limited quantity issued per person. * * As a limit to the total amount of issuance per person, it is * common to assign 1 to limit duplicate issuance to the same citizen, * or to use the NULL value to set no limit. * * Of course, by assigning a value of N, the total amount issued * to the same citizen can be limited. */ volume_per_citizen: null | (number & tags.Type<"uint32">); /** * Expiration day(s) value. * * The concept of expiring N days after a discount coupon ticket is issued. * * Therefore, customers must use the ticket within N days, if possible, * from the time it is issued. */ expired_in: null | (number & tags.Type<"uint32">); /** * Expiration date. * * A concept that expires after YYYY-MM-DD after a discount coupon ticket * is issued. * * Double restrictions are possible with expired_in, of which the one * with the shorter expiration date is used. */ expired_at: null | (string & tags.Format<"date-time">); }

Describe what value must be filled in the property.

When documenting DTO properties, concentrate on what value must be filled in the property. In other words, describe the property’s purpose and the value’s meaning with its domain restrictions. IShoppingCouponRestriction is a good example describing the domain restrictions.

Also, if target property is a reference to another entity by foreign key, it would better to utilizing the {@link target} tag. Looking at above example type IShoppingCartCommodityStock, it has a property stock_id which is a foreign key to the IShoppingSaleUnitStock entity with {@link target} tag.

Namespace Strategy

undefined

IBbsArticle.ts
import { tags } from "typia"; /** * Article entity. * * `IBbsArticle` is an entity representing an article in the BBS (Bulletin Board System). */ export interface IBbsArticle extends IBbsArticle.ICreate { /** * Primary Key. */ id: string & tags.Format<"uuid">; /** * Creation time of the article. */ created_at: string & tags.Format<"date-time">; /** * Last updated time of the article. */ updated_at: string & tags.Format<"date-time">; } export namespace IBbsArticle { /** * Information of the article to create. */ export interface ICreate { /** * Title of the article. * * Representative title of the article. */ title: string; /** * Content body. * * Content body of the article writtn in the markdown format. */ body: string; /** * Thumbnail image URI. * * Thumbnail image URI which can represent the article. * * If configured as `null`, it means that no thumbnail image in the article. */ thumbnail: | null | (string & tags.Format<"uri"> & tags.ContentMediaType<"image/*">); } /** * Information of the article to update. * * Only the filled properties will be updated. */ export type IUpdate = Partial<ICreate>; }

typia.llm.application<App, Model>() and HttpLlm.application() function copy the parent namespaced type description comment to the children types. @agentica calls this comment writing strategy as namespace documentation, and it is recommended for the efficient documentation.

As you can see from the above example, BbsArticleService has many CRUD functions about the IBbsArticle namespaced type. By the way, the above IBbsArticle.ICreate and IBbsArticle.IUpdate types are not repeating same description comments about the IBbsArticle type. Instead, just writing the short description comment about them, and just compose the LLM function calling application schema.

In that case, the IBbsArticle type’s description comment would be copied to the IBbsArticle.ICreate and IBbsArticle.IUpdate types like above “Console Output” case. It’s a good strategy to avoid repeating same description comments, and also to deliver enough information to the LLM function calling.

Domain Driven Development

Document Driven Development is the AI agent ​​version of Domain Driven Development.

There’s a development methodology called “Domain Driven Development”. It separates complicate project to small domains, and concentrate on each domain independently. It’s because the complexity of the project is not from the project itself, but from the interaction between domains.

And such Domain Driven Development approach is also applied to the AI agent development, and I call it as “Document Driven Development”. In fact, Document driven development is a pun on the AI ​​agent word for Domain Driven Development.

Let’s separate the agent’s functionality into small functions, and concentrate on each function independently. Then, the complexity of the agent will be reduced, and the success rate will be independent for each function. It’s the same as the Domain Driven Development methodology.

Last updated on