Introduction
As we setup the stage in Part 3 we are ready to continue exploration. Next is SWE-1.
Prerequisites
- SWE-1 is selected as a model
- The following prompt is run:
Create a test case utilizing provided constants for navigating to the web app, login, create/edit/delete an article. Try to verify the result after every major step. Use provided instructions
Generated POM (Page Object Model)
class ConduitApp {
constructor(private page: Page) {}
// Locators
private get signInLink() { return this.page.getByRole('link', { name: 'Sign in' }); }
private get emailInput() { return this.page.getByRole('textbox', { name: 'Email' }); }
private get passwordInput() { return this.page.getByRole('textbox', { name: 'Password' }); }
private get signInButton() { return this.page.getByRole('button', { name: 'Sign in' }); }
private get newArticleLink() { return this.page.getByRole('link', { name: 'New Article' }); }
private get articleTitleInput() { return this.page.getByRole('textbox', { name: 'Article Title' }); }
private get articleAboutInput() { return this.page.getByRole('textbox', { name: 'What\'s this article about?' }); }
private get articleContentInput() { return this.page.getByRole('textbox', { name: 'Write your article (in markdown)' }); }
private get publishArticleButton() { return this.page.getByRole('button', { name: 'Publish Article' }); }
private get editArticleButton() { return this.page.getByRole('link', { name: 'Edit Article' }).first(); }
private get deleteArticleButton() { return this.page.getByRole('button', { name: 'Delete Article' }).first(); }
private get articleTitle() { return this.page.locator('h1'); }
private get articleContent() { return this.page.locator('div.article-content > p'); }
private get articleAuthor() { return this.page.locator('.author'); }
private get articleMeta() { return this.page.locator('.article-meta'); }
// Actions
async goto() {
await this.page.goto(url);
}
async login(email: string, password: string) {
await this.signInLink.click();
await this.emailInput.fill(email);
await this.passwordInput.fill(password);
await this.signInButton.click();
}
async createArticle(title: string, about: string, content: string) {
await this.newArticleLink.click();
await this.articleTitleInput.fill(title);
await this.articleAboutInput.fill(about);
await this.articleContentInput.fill(content);
await this.publishArticleButton.click();
}
async updateArticle(newTitle: string, newAbout: string, newContent: string) {
await this.editArticleButton.click();
await this.articleTitleInput.fill(newTitle);
await this.articleAboutInput.fill(newAbout);
await this.articleContentInput.fill(newContent);
await this.publishArticleButton.click();
}
async deleteArticle() {
await this.deleteArticleButton.click();
}
}
NOTE The provided example was generated from the first time. Updates which were needed to be made were to combine all tests (there were separate test for each action, which leads to failing tests due to single Log in), to remove one assertion for created/edited article ('await expect(app.articleAuthor).toContainText(email.split('@')[0]);'), to add .first() for the delete button and edit button, and to delete wrong navigation to article page. There are still unused locators, which were defined.
Pros of the selected pattern
- Encapsulation: Prevents direct access from outside the class, enforcing usage only within class methods.
- Cleaner API: Exposes only actions, not locators, to test code.
- Reduces Misuse: Prevents test code from making direct assertions on locators.
Cons of the selected pattern
- Test Flexibility: Makes it harder to write custom assertions or interact directly with elements from the test.
- Discoverability: Less transparent for someone reading the test and wanting to know what elements are available.
What's next?
Stay tuned for next article in which we will explore xAI-Grok-3.
Please, do not hesitate to start conversation regarding the test or it's result.
🙏🏻 Thank you for reading! Building robust, scalable automation frameworks is a journey best taken together. If you found this article helpful, consider joining a growing community of QA professionals 🚀 who are passionate about mastering modern testing.
Join the community and get the latest articles and tips by signing up for the newsletter.
Top comments (2)
pretty cool seeing the ai spit out so much boilerplate in one go, kinda makes me wonder- you think letting ai handle more of this test logic will actually make things smoother long-term or just cause new headaches down the line?
The whole purpose of this experiment is to test and show current state of ai created tests. The two main components, which are responsible for the quality of the created test are actual LLM’s capabilities and the context you provide. We can’t control the first part, but we can provide better context and instructions what and how we want. Here comes the Playwright MCP server. Before that we can only instruct the models by text input, and now we can actually give the LLM capabilities of explore the AUT (application under test) themselves and gain better context. Here I tried to compare different LLM with simple and equal text content and the ability to actually explore the AUT. I have planned deeper research on selected LLM, which have shown promising results in near future . Keep in mind that this is the very beginning of the journey.