AI Agent assisted Coding

I recently did my first longer coding sessions with an AI agent, and I wanted to share my experience and to recep for my own understanding of the process.

The first attempt was doing the GitHub tutorial.

• "Build applications with GitHub Copilot agent mode" via: continuous-copilot

Until then I had used Copilot in the editor and on the command line, but primarily to ask about syntex or my favorite feature: explain. I had used the agent to generate some code, but I had not really used it to build a complete application, just some SQL. Here I had switched to using Claude Sonnet and compared to the default, it had actually worked.

I decided to give agent mode a more extensive go, since I had seen some interesting demos and videos on YouTube and I wanted to try it out myself.

I started with the tutorial, which was a good introduction to the capabilities of the AI agent. The tutorial guided me through the process of setting up the agent and using it to build a complete application.

Things started out smoothly and I made good progress fast, but at I certain point, I went into a kind of loop, not being able to get out of the changes made by itself, attempting to get the infrastructure and dependencies in place it kept, stepping on it's own toes, so to speak - in the end I ran out of time and had to stop.

A few days later, I revisited the topic and decided to try again, this time with a more simple approach. Instead of a complete and somewhat complex application, I wanted to build a simple web application and I had just the idea.

I am playing this game called Partners with some friends of mine. It is sort of a advanced Ludo game and very competitative and fun. We collect the scores and keep track of the results - if this does not say web application I do not know what does, yes some people would hear spreadsheet.

We are 6 players, divided into 3 teams of 2 players each. After each game, we change teams and each individual gets a score per game and therefor the total is interesting and doing some presentation and statistics would be useful.

I had started to outline a JSON file to describe the data structure as I wanted to implement this application just, but teaming up with a copilot agent I expected to make faster progress, since I am by no means a web developer.

First up, I set up a repository and copied in the description of the application from the tutorial mentioned above:

• "Build applications with GitHub Copilot agent mode" via: continuous-copilot

I edited it to contain the description of my application.

• jonasbn/partners-competition-app: partners_app_outline

I copied my existing JSON file into the repository and then I started the agent mode in Visual Studio Code.

• jonasbn/partners-competition-app: data

I asked the agent to help me build a web application that would allow me to input the scores from the game and display the results in a nice way.

And it did so, quite quickly.

I made it create a historic table of the games played and it looked good. Next up was the statistics, when asking for various statistics, the agent would suggest various charts and graphs, which I found quite useful. It did not take long before I had a working application and useful application, which I could use to display the games, scores and results.

I decided to give it a human touch, the application was a bit too sterile and bootstrappy. I decided to generate some avatar images for the players. I did that in ChapGPT and after various interations some image manipulation, I had some nice images to use.

This is where I believe we still need a human at the keyboard. Yes we should review the code and the suggestions made by the AI agent, but we also need to add the human touch to the application. I must admit I do not understand all of the implementation details and suggestions made by the agent, but I can always ask and as said, explain is my favorite feature of Copilot.

The avatarts was added quite easily, but I wanted something more, so based on the ranking in the games and in total, I wanted the application to vary the avatar images, so all players had 3 different images, portraying 3 different moods, depending on the ranking in the game and overall.

• Happy
• Neutral
• Sad

Finding my own suggestion incredibly amusing, I decided that I wanted to the images to pop-up when hovering over the avatar image so the user would not miss how funny I am, so I asked the agent to help me implement this.

And then things started to go sideways and I ended up in a situation like when I did the GitHub tutorial, where the agent made changes to the code and had to correct itself or lost track.

After many iterations and problems with layers, z-indices and containers, I finally got the pop-up working as expected.

In addition I also quite easily got it to add some cards to the top of the page of the application to highlight some aspects of the game data. This was quite easy, the challenge here is to know the proper terms. Since I am not a web designer or developer, I did some googling on the side to find the proper terms, so I could ask the agent for help in a more precise manner.

Adding support for Danish and English was also quite easy, I just had to ask the agent to help me implement this, and it did so, without any problems and adding light and dark mode was also a breeze, I just had to ask the agent to help me implement this, and it did so, without any problems.

I had based the data on the first games played, so when I got the next batch I got the agent to add the data - easy peasy.

To recap the implementation and experience:

• The application is awesome and I am very pleased with the result, yes there is room for improvement, but who said we where done?

• The repository requires some clean up, there are a few files for testing, debugging and documentation introduced by the agent, which are not needed in the final application

• The AI agent was very helpful and made good suggestions, I am very impressed with the capabilities in regards to grasping the concept of the game and suggesting statistics and visualizations, I did ask it to remove some, but luckily that was easy to do

• The application often lost track of it's integration with the editor and indicated that the port was already in use, which it was, by itself. So a lot of time was spent just getting the tools to interact properly

• Much of the code generated by the agent was revisited in a later iteration, and the AI has no remorse or is afraid to kill it's darlings, the agent was quite helpful in explaining the challenges in regards to z-index and layering. I read up on some other resources, but in general it was very helpful and educating and the agent's explanations were spot on compared to the resources I found

• I have almost written no code myself, I made a few manual changes in when the z-index problem was at it's worst, in retrospect I should just have interacted with the agent

Take-aways:

• Be sure to have a good outline of the application goal and concept, you need context for your agent and context is key
• A data structure can get you along way (yes you can have the agent generate this), but it also let's you think about the foundation of the application, for me this goes hand in hand with the outline
• Be direct and do not accept all suggestions or accept them but do not hestitate to request changes and refine the suggestions. I learned this over time, so it is a part of getting to learn to interact with the agent
• Do not be afraid to ask the agent to explain the code, it is very helpful and educational
• Be patient, the agent is and take it easy
• Step away from the session and think about the best way to proceed and interact with the agent, just like you would do if coding it by hand yourself
• I need to be better to commit changes to the repository each time something works, this is how you would do it if coding by hand - I simply forgot to do this, since I was so focused on the interaction with the agent

The applicataion is available on GitHub:

• jonasbn/partners-competition-app

And is deployed on Digital Ocean App Platform:

• Digital Ocean: partners-competition-app

This is not the last time I will use agent mode. I have some candidates where I could do with agent assistance. I also have to finish the GitHub tutorial, it might require a restart, but that is okay, I learned a lot from the second attempt with my own application and I can apply that to the tutorial.

Please share your experiences with AI agents in the comments, I would love to hear about your experiences and how you have used AI agents to assist you in your coding projects, what works and what does not - how do we avoid the loops and pitfalls and how do we get the tools to work in unison without too much hassle.