DEV Community

Cover image for DynamoDB with Enhanced Client - Schemas and data serialization made easy
Neel
Neel

Posted on • Edited on

DynamoDB with Enhanced Client - Schemas and data serialization made easy

#java #aws #springboot #dynamodb

DynamoDB is a powerful NoSQL Database offered and managed by AWS. Flexible schema is a key feature offered by NoSQL systems and DynamoDB is no exception. Even though flexible schemas may sound appealing, it is better to keep the schema in control to ensure the tables don't go out of control with different documents having different fields with no correlation

The AWS SDK-v2 for Java offers a high level API called the EnhancedClient to setup a strict table schema for the documents and ensure consistent serialization and de-serialization of data across the application. Let's see this in action

I assume that the readers already know their way around Spring Boot and DynamoDB, so I am skipping the bare minimum parts like table creation and stuffs

Blueprint

I will be using DynamoDB Local for this demo. Ensure that you run the local JAR with sharedDb mode, otherwise the local DB will create different accounts for different clients.

The app will have the following endpoints to perform the Create and Read operations

  • POST /api/v1/book
  • GET /api/v1/book/id
  • GET /api/v1/books
  • GET /api/v1/books/year/{year_published}

The following is going to be the structure of a document to be stored in the Books table

  • id - Partition Key
  • We will add a new GSI with the yearPublished as the Partition Key 
{
  "id": 1,
  "title": "Chip War",
  "description": "The Fight for the World's Most Critical Technology",
  "isbn": 9781982172008,
  "author": {
    "name": "Chris Miller",
    "biography": "Historian and book author",
    "otherBooksPublished": [
      {
        "isbn": 9780316057011,
        "title": "The Real Animal House"
      }
    ]
  },
  "yearPublished": 2022,
  "languagesAvailableIn": ["English", "Spanish", "German"],
  "currentEdition": 1,
  "averageRating": 4.44,
  "genere": ["science", "politics"],
  "createdAt": 1749283110
}
Enter fullscreen mode Exit fullscreen mode

Notice that I have added nested fields and different types of data specifically to show the schema mapping using the EnhancedClient API

Lets Cook!

DynamoDB is running as a docker container on my machine and I have already bootstrapped a new springboot project with just spring-web and lombok as the dependencies.

Client Setup

To setup the EnhancedClient API, the following dependencies should be added to the project

software.amazon.awssdk:dynamodb:2.31.58
software.amazon.awssdk:dynamodb-enhanced:2.31.58
Enter fullscreen mode Exit fullscreen mode

The following code shows the basic setup for a singleton Enhanced Client pointing to the local DynamoDB instance running on port 8000. To connect to the actual DynamoDB instance on AWS, the endpointOverride should be removed and the credentials should be configured based on your application setup. Refer this for more information

package com.itassistors.books;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedClient;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;

import java.net.URI;

@Configuration
public class DynamoConfig {
    @Bean
    DynamoDbEnhancedClient dynamoDbEnhancedClient() {
        DynamoDbClient dbClient = DynamoDbClient.builder()
                .region(Region.US_EAST_1)
                .endpointOverride(URI.create("http://localhost:8000")) // Use the local DynamoDB endpoint
                .build();

        return DynamoDbEnhancedClient.builder()
                .dynamoDbClient(dbClient)
                .build();
    }
}
Enter fullscreen mode Exit fullscreen mode

Bean and TableSchema<T> setup

The beauty of using the EnhancedClient API is the control you get over the schema of the table and robust de-/serialization of the data stored in DynamoDB. We will create the required classes in Java to handle the data similar to the sample JSON data mentioned above, starting with the Book.java class, Author.java and OtherBooksPublished.java for the nested data

package com.itassistors.books.entity;

import lombok.AllArgsConstructor;
import lombok.Data;
import software.amazon.awssdk.enhanced.dynamodb.EnhancedType;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;
import software.amazon.awssdk.enhanced.dynamodb.mapper.StaticAttributeTags;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbBean;

import java.util.List;

@AllArgsConstructor
@Data
@DynamoDbBean
public class Book {
    private Long id;
    private String title;
    private String description;
    private Long isbn;
    private Author author;
    private Integer yearPublished;
    private List<String> languagesAvailableIn;
    private Integer currentEdition;
    private Double averageRating;
    private List<String> genres;
    private Long createdAt;

    public Book() {
    }

    public static final TableSchema<Book> TABLE_SCHEMA = TableSchema.builder(Book.class)
            .newItemSupplier(Book::new)
            .addAttribute(Long.class, a -> a.name("id")
                    .getter(Book::getId)
                    .setter(Book::setId)
                    .tags(StaticAttributeTags.primaryPartitionKey()))
            .addAttribute(String.class, a -> a.name("title")
                    .getter(Book::getTitle)
                    .setter(Book::setTitle))
            .addAttribute(String.class, a -> a.name("description")
                    .getter(Book::getDescription)
                    .setter(Book::setDescription))
            .addAttribute(Long.class, a -> a.name("isbn")
                    .getter(Book::getIsbn)
                    .setter(Book::setIsbn))
            .addAttribute(EnhancedType.documentOf(Author.class, Author.TABLE_SCHEMA), a -> a.name("author")
                    .getter(Book::getAuthor)
                    .setter(Book::setAuthor))
            .addAttribute(Integer.class, a -> a.name("yearPublished")
                    .getter(Book::getYearPublished)
                    .setter(Book::setYearPublished)
                    .addTag(StaticAttributeTags.secondaryPartitionKey("yearPublishedIndex")))
            .addAttribute(EnhancedType.listOf(String.class), a -> a.name("languagesAvailableIn")
                    .getter(Book::getLanguagesAvailableIn)
                    .setter(Book::setLanguagesAvailableIn))
            .addAttribute(Integer.class, a -> a.name("currentEdition")
                    .getter(Book::getCurrentEdition)
                    .setter(Book::setCurrentEdition))
            .addAttribute(Double.class, a -> a.name("averageRating")
                    .getter(Book::getAverageRating)
                    .setter(Book::setAverageRating))
            .addAttribute(EnhancedType.listOf(String.class), a -> a.name("genres")
                    .getter(Book::getGenres)
                    .setter(Book::setGenres))
            .addAttribute(Long.class, a -> a.name("createdAt")
                    .getter(Book::getCreatedAt)
                    .setter(Book::setCreatedAt))
            .build();
}
Enter fullscreen mode Exit fullscreen mode 
package com.itassistors.books.entity;

import lombok.Data;
import software.amazon.awssdk.enhanced.dynamodb.EnhancedType;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;

import java.util.List;

@Data
public class Author {
    private String name;
    private String biography;
    private List<OtherBooksPublished> otherBooksPublished;

    public static final TableSchema<Author> TABLE_SCHEMA = TableSchema.builder(Author.class)
            .newItemSupplier(Author::new)
            .addAttribute(String.class, a -> a.name("name")
                    .getter(Author::getName)
                    .setter(Author::setName))
            .addAttribute(String.class, a -> a.name("biography")
                    .getter(Author::getBiography)
                    .setter(Author::setBiography))
            .addAttribute(EnhancedType.listOf(EnhancedType.documentOf(OtherBooksPublished.class, OtherBooksPublished.TABLE_SCHEMA)),
                    a -> a.name("otherBooksPublished")
                            .getter(Author::getOtherBooksPublished)
                            .setter(Author::setOtherBooksPublished))
            .build();
}
Enter fullscreen mode Exit fullscreen mode 
package com.itassistors.books.entity;

import lombok.Data;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;

@Data
public class OtherBooksPublished {
    private Long isbn;
    private String title;

    public static final TableSchema<OtherBooksPublished> TABLE_SCHEMA = TableSchema.builder(OtherBooksPublished.class)
            .newItemSupplier(OtherBooksPublished::new)
            .addAttribute(Long.class, a -> a.name("isbn")
                    .getter(OtherBooksPublished::getIsbn)
                    .setter(OtherBooksPublished::setIsbn))
            .addAttribute(String.class, a -> a.name("title")
                    .getter(OtherBooksPublished::getTitle)
                    .setter(OtherBooksPublished::setTitle))
            .build();
}
Enter fullscreen mode Exit fullscreen mode

I have added the TableSchema built for all the three classes as constant static variables for ease of use and to ensure those are immutable

A few key things to note here are…

  • The newItemSupplier attribute is required to create new instances of the class for de-seralizing the JSON data fetched from DynamoDB

  • For all the required attributes stored in the table, the appropriate getters and setters should be setup using the addAttribute method. If a particular attribute of the class is not stored to the DB, then such fields could be ignored from the TableSchema

  • Key fields must be attributed using the addTag method to let the client know on how to handle those fields. I have added the tags for the primary partitionKey - id, the GSI partitionKey - yearPublished. For sort keys, the same method can be used with the primarySortKey method

  • If the data stored in your table is going to be flat without any nesting, then you need not go through the process of creating explicit table schemas. It can be achieved easily using TableSchema.fromBean(Bean.class), but that’s not the case here

    • To handle nested or complex types such as lists and objects, you can make use of the EnhancedType.documentOf and EnhancedType.listOf wherever required. You need to ensure that you handle the enhanced types properly to avoid any de-serialization errors

Service Layer

Lets create a service class to power the APIs. We will start with injecting the DynamoDB enhanced client and the methods for working with the DB

package com.itassistors.books;

import com.itassistors.books.entity.Book;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import software.amazon.awssdk.core.pagination.sync.SdkIterable;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedClient;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbIndex;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbTable;
import software.amazon.awssdk.enhanced.dynamodb.Key;
import software.amazon.awssdk.enhanced.dynamodb.model.Page;
import software.amazon.awssdk.enhanced.dynamodb.model.PageIterable;
import software.amazon.awssdk.enhanced.dynamodb.model.QueryConditional;
import software.amazon.awssdk.enhanced.dynamodb.model.QueryEnhancedRequest;

import java.time.Instant;
import java.util.List;

@Slf4j
@Service
public class BookService {
    private final DynamoDbTable<Book> booksTable;

    public BookService(@Value("${db.table.books:books}") String booksTableName,
                       DynamoDbEnhancedClient dynamoDbEnhancedClient) {
        this.booksTable = dynamoDbEnhancedClient.table(booksTableName, Book.TABLE_SCHEMA);
    }

    public List<Book> getAllBooks() {
        PageIterable<Book> results = booksTable.scan();

        log.info("Fetched {} books from the database", results.stream().count());
        return results.items().stream().toList();
    }

    public Book addBook(Book book) {
        book.setId(Instant.now().getEpochSecond());
        book.setCreatedAt(System.currentTimeMillis());

        booksTable.putItem(book);
        log.info("Added book: {}", book.getTitle());

        return book;
    }

    public Book getBookById(Long id) {
        log.info("Fetching book by ID: {}", id);
        Key key = Key.builder().partitionValue(id).build();
        return booksTable.getItem(key);
    }

    public List<Book> getBookByYearPublished(Integer yearPublished) {
        DynamoDbIndex<Book> isbnIndex = booksTable.index("yearPublishedIndex");
        log.info("Querying book by year published: {}", yearPublished);

        QueryConditional queryConditional = QueryConditional.keyEqualTo(Key.builder()
                .partitionValue(yearPublished)
                .build());
        QueryEnhancedRequest queryEnhancedRequest = QueryEnhancedRequest.builder()
                .queryConditional(queryConditional)
                .build();

        SdkIterable<Page<Book>> results = isbnIndex.query(queryEnhancedRequest);
        log.info("Found {} books published in the year {}", results.stream().count(), yearPublished);
        if (results.stream().findAny().isEmpty()) {
            log.warn("No books found for the year {}", yearPublished);
            return List.of();
        }

        List<Book> books = results.stream()
                .flatMap(page -> page.items().stream())
                .toList();

        log.info("Returning {} books published in the year {}", books.size(), yearPublished);
        return books;
    }
}
Enter fullscreen mode Exit fullscreen mode

In the above code, I have created a single instance for the DynamoDB table

this.booksTable = dynamoDbEnhancedClient.table(booksTableName, Book.TABLE_SCHEMA);

The table instances are supposed to be immutable and are eligible singleton candidates. For this simple application, this setup is sufficient, but for large scale applications you need to create a singleton for instantiating the table objects that can be used across the entire application without re-creating it

The rest of the code should be pretty self explanatory. We use the DB methods to perform the scan and query operations. The key aspect of this setup is that, you don’t need to go through any implicit data serializtion or de-seralization, and those things are handled behind the scenes consistently with the use of the defined TableSchema’s

App In Action

The controller is just going to relay the requests to the service layer

package com.itassistors.books;

import com.itassistors.books.entity.Book;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.util.List;

@Slf4j
@RestController
@RequestMapping("/api/v1")
public class BookController {

    private final BookService bookService;

    public BookController(BookService bookService) {
        this.bookService = bookService;
    }

    @GetMapping("/books")
    public List<Book> getBooks() {
        log.info("Fetching all books");
        return bookService.getAllBooks();
    }

    @PostMapping("/book")
    public Book addBook(@RequestBody Book book) {
        log.info("Adding a new book: {}", book.getTitle());
        return bookService.addBook(book);
    }

    @GetMapping("/book/{id}")
    public ResponseEntity<Book> getBookById(@PathVariable("id") Long id) {
        log.info("Fetching book by id: {}", id);
        Book bookById = bookService.getBookById(id);
        if (bookById == null) {
            log.error("Book with ID {} not found", id);
            return ResponseEntity.notFound().build();
        }

        return ResponseEntity.ok(bookById);
    }

    @GetMapping("/books/year/{yearPublished}")
    public List<Book> getBooksByYearPublished(@PathVariable("yearPublished") Integer yearPublished) {
        log.info("Fetching books published in the year: {}", yearPublished);
        return bookService.getBookByYearPublished(yearPublished);
    }
}
Enter fullscreen mode Exit fullscreen mode

Image description

Conclusion

Using the EnhancedClient API solves the major pain-point of data serialization and it also enforces a strict, yet a flexible schema for the tables. One might find it boring to write the TableSchema for the beans, and I’m one. So I delegate the task of generating the table schemas to copilot. ChatGPT also does a better job of doing this provided you give it a tailored prompt

Happy Coding!

Top comments (0)

Some comments may only be visible to logged-in visitors. Sign in to view all comments. Some comments have been hidden by the post's author - find out more