Ryan Nazareth

Posted on May 19

Building a Sports Marketing Video Generator using Nova Canvas and Nova Reel

#aws #bedrock #streamlit #ecs

Introduction

In today's fast-paced digital marketing landscape, creating engaging sports content quickly is essential. Traditional video production is time-consuming and expensive, often requiring specialized skills and equipment. What if marketers could generate professional sports marketing videos from a single image with just a few clicks?

In this blog post, we'll explore a solution called that builds a streamlit application hosted n ECS and leverages Amazon Nova's generative AI capabilities to transform sports images into dynamic marketing videos.

The Business Challenge

Marketing teams across industries face common challenges:

Limited resources for video production
Need for rapid content creation across multiple channels
Maintaining brand consistency across all visual assets
Scaling content production without proportionally scaling costs

This solution addresses these challenges by providing an intuitive web interface that allows non-technical users to generate professional-quality video content from static images with just a few clicks.

Architecture

The Canvas Video application is built on a modern, serverless architecture that leverages several AWS services.

App: Dockerised streamlit application deployed in AWS ECS (Fargate) as a service, with tasks running in multiple availability zones to ensure high availability.
Authentication: Amazon Cognito user pool with username and password for authentication.
Load Balancer: ALB intercepts requests, redirects unauthenticated users to Cognito for authentication, and then forwards authenticated users to the backend application with user claims, enabling secure access to the app running as ECS tasks.
AI Services: AWS Bedrock (Nova Reel, Nova Canvas) and Amazon Rekognition
Storage: S3 for storing the videos generated from Nova Reel.

The basic flow is as follows. A user navigates to url alias record in Route53 and authenticates via Cognito. User uploads a sports image. AWS Rekognition analyzes the image to confirm it is a sports-related image. The user has the option of editing the image using Nova Canvas by providing inpainting/outpainting related prompts. Nova Reel generates a sports marketing video based on the image and prompt. The video is stored in S3 and a presigned URL is provided to the user in the streamlit application.

Key Features

1. Sports Image Classification

The app uses Amazon Rekognition to analyze uploaded images and determine if they're sports-related. It can identify specific sports like basketball, football, soccer, etc.

def is_sports_image(self, image_bytes):
    """Determine if the image is sports-related using Amazon Rekognition"""
    try:
        rekognition = boto3.client('rekognition')
        response = rekognition.detect_labels(Image={'Bytes': image_bytes})

        # Extract labels from the response
        labels = [label['Name'].lower() for label in response['Labels']]

        # Determine the specific sport type from labels
        sport_type = self.determine_sport_type(labels, response['Labels'])

        # Check if any sports keywords are in the labels
        for keyword in self.sports_keywords:
            if keyword.lower() in labels:
                return True, labels, sport_type

        # Check for confidence scores on sports-related activities
        for label in response['Labels']:
            if any(keyword.lower() in label['Name'].lower() for keyword in self.sports_keywords):
                if label['Confidence'] > 70:
                    return True, labels, sport_type

        return False, labels, "General Sports"

    except Exception as e:
        logger.error(f"Error in sports image classification: {str(e)}")
        return False, [], "General Sports"

2. Image Editing with Nova Canvas

Before generating a video, users can enhance their images using AWS Bedrock Nova Canvas. The app supports:

Inpainting: A technique used in image processing to fill in missing or damaged parts of an image in a way that blends seamlessly with the surrounding areas. This process can reconstruct removed elements or extend the background of an image while preserving the visual coherence of textures, colors, and patterns.
Outpainting: A technique used to expand an image beyond its original borders by generating new visual content that seamlessly extends the existing scene. Unlike inpainting, which fills in missing areas within an image, outpainting creatively imagines what might lie beyond the current frame, effectively "uncropping" it to add context or detail.

The method implementing this logic is included in the snippet below.
The mask prompt tells Canvas which parts of the image to edit or preserve For inpainting, the mask is what you want to change
whilst dor outpainting the mask is what you want to keep.

def process(self, image_bytes, negative_prompt, main_prompt, mask_prompt, operation_type, config=None):
    """Process image using Amazon Nova Canvas for inpainting or outpainting with sports focus"""
    try:
        # Check if the image is sports-related
        is_sports, labels = self.sports_classifier.is_sports_image(image_bytes)

        if not is_sports:
            return "NOT_SPORTS_IMAGE"

        # Convert image bytes to base64
        image_base64 = base64.b64encode(image_bytes).decode('utf-8')

        # Use default config if none provided
        if config is None:
            config = DEFAULT_IMAGE_CONFIG

        body = {
            "taskType": operation_type,
            "imageGenerationConfig": config
        }

        # Add the appropriate parameters based on operation type
        if operation_type == "INPAINTING":
            body["inPaintingParams"] = {
                "text": main_prompt,
                "maskPrompt": mask_prompt,
                "negativeText": negative_prompt,
                "image": image_base64,
            }
        elif operation_type == "OUTPAINTING":
            body["outPaintingParams"] = {
                "text": main_prompt,
                "maskPrompt": mask_prompt,
                "negativeText": negative_prompt,
                "image": image_base64,
            }

        response = self.bedrock_runtime.invoke_model(
            modelId="amazon.nova-canvas-v1:0",
            body=json.dumps(body),
            accept=self.accept, contentType=self.content_type
        )
        response_body = json.loads(response.get("body").read())
        base64_image = response_body.get("images")[0]
        base64_bytes = base64_image.encode('ascii')
        image_bytes = base64.b64decode(base64_bytes)

        return image_bytes

    except Exception as e:
        logger.error(f"Error in Nova Canvas processing: {str(e)}")
        return None

3. Video Generation with Nova Reel

The core feature is video generation using AWS Bedrock Nova Reel. This generates dynamic videos using using the context provided in the image and a video prompt that is autogenerated in the backend. Users can select from different marketing styles to influence/update the prompt:

Dynamic action
Athlete showcase
Team spirit
Fan experience
Product in action
Inspirational

The video generation request to Amazon Nova Reel is invoked asynchronously using the context provided in the image and the results are stored in Amazon S3 for later retrieval. The video object is accessible by the user in the frontend through an auto generated presigned url.

Note Amazon Nova currently only allows a video duration of 6 secs if using text prompt with an Image.

4. Prompt Enhancement

The app uses a base prompt template and enhances it with sport-specific details:

def enhance_prompt(self, marketing_prompt, brand=None, sport_type=None):
    """Enhance the marketing prompt with the base Nova Reel prompt"""
    enhanced_prompt = NOVA_REEL_BASE_PROMPT.strip() + " " + marketing_prompt

    if brand:
        enhanced_prompt += f" for {brand}"

    if sport_type:
        enhanced_prompt += f" in the context of {sport_type}"

    return enhanced_prompt

Deploying the application to AWS

To deploy this solution in your AWS environment, first clone the repository and install the python dependencies.

$ git clone https://github.com/ryankarlos/llm-use-cases.git
$ cd image_and_video

$ python -m venv venv
$ . venv/bin/activate

pip install -r requirements.txt
pip install -r src/image_and_video/requirements.txt

Now we will need to deploy the resources in AWS shown in the architecture diagram using terragrunt and terraform. You can install it here and here

The inputs and locals block in terragrunt.hcl file in the terragrunt folder has a list of default variables set which will be passed to the terraform scripts during deployment of the resources.
You can update some of the variables listed below (the email and username will need to be updated to your username and email so you can reset your password set by Cognito

locals {
  region = "us-east-1"
  username = <replace with your own username for cognito>
  email = <replace with your own email>
  ecr_repo_name = "canvas-video"
}

inputs = {
  hosted_zone_name = "awscommunity.com"
  ....
  bucket_name = "image-llm-example"
  subdomain = "nova-video"
}

Now we will run terragrunt commands following your installation to generate a plan of the resources to be deployed and then apply the changes.

$ cd terragrunt
$ terragrunt plan
$ terragrunt apply

Once the resources are deployed, we will need to build and push the docker image with the app code, to ECR, which will be automatically deployed to the ECS service. Execute the ecr-build-push.sh script which will handle the Docker build and push process. Note that this uses the defaults for region and ecr repo name. If you have selected different values, then you will need to update these defaults or set the environment variables accordingly.

# Configuration
AWS_REGION=${AWS_REGION:-"us-east-1"}
ECR_REPOSITORY=${ECR_REPOSITORY:-"canvas-video"}
IMAGE_TAG=${IMAGE_TAG:-"latest"}

Once the image is pushed, a manual step will need to be performed to deploy the task in ECS. Navigate to the ECS console, and navigate to the service in the cluster that was deployed. Click on the Update Service option and select force new deployment.

Wait for a few minutes for the deployment. The tasks will change status from PROVISIONING to PENDING to ACTIVATING and if successful finally show as RUNNING status.

Navigate to the Route53 url and you should be able to see the Cognito login page. Enter your username and temp password sent to your email address, which will prompt for a password reset. Reset your password and you should now see the streamlit application home page.

Using the Application

Upload the image using the upload image tab. This app uses Amazon Rekognition behind the scenes to label the image. If the image is not sports-related, it will throw an error.

If the image is sports-related, you will see the image uploaded and the tags. The app automatically identifies elements like "basketball," "soccer," or "athlete" in your image, helping tailor the marketing content to your specific sport.

In the example below, an image of a tennis racquet with balls on a clay court has been uploaded.

Image Processing options

In the left tab, you will see different options for performing inpainting, outpainting, or no processing on the image

No Processing: Use your image as-is for video generation
Inpainting: Replace or modify specific areas within your image. Perfect for adding brand elements or removing unwanted objects. For example in the image above, we can ask to replace the yellow balls (mask) with striped balls (processing prompt). We can set the negative prompt to blur to prevent any blurring.

Outpainting: Extend your image beyond its original boundaries. Great for creating more dynamic compositions or adding space for text. In the image above, we can set the mask as the racquet and balls, and set the processing prompt to garden to show an image of the racquet and balls in a garden instead of a clay court.

If you choose inpainting or outpainting, you'll need to provide:

A main prompt describing what you want to add or modify.
A mask prompt indicating which area to modify
An optional negative prompt to specify what to avoid

Image Video Generation

In the sidebar, you'll find several options to customize your marketing video under the Video Settings options. Choose from templates in the dropdown menu for Marketing Video Style like "Dynamic Action," "Team Spirit," or "Product in Action" to set the tone of your video. You can also ddd your brand name to personalize the marketing message.

The app uses these selections to craft a specialized prompt for the AI video generator, optimizing it for sports marketing content.
The "Review Final Prompt" expander allows you to see and edit this final prompt. This is helpful if you want to fine-tune specific aspects of your marketing message. You can experiment with different combinations of image processing, marketing styles, and sport types to create the perfect sports marketing video for your brand.

Once you're satisfied with your settings, click the "Generate Sports Marketing Video" button. The system will create a specialized marketing prompt based on your selections and generate a dynamic sports marketing video using AWS Bedrock Nova Reel During generation, you'll see progress updates. This process typically takes a 3-5 mins as the AI creates your custom video. The video is stored securely in your AWS S3 bucket that you selected when deploying the terraform infrastructure.

It is also accessible from the frontend through a presigned URl.

Future Improvements

There are several ways this application could be enhanced:

Multi-image input - Allow users to upload multiple images for more dynamic videos
Custom audio - Add options for background music or voiceovers
Video templates - More specialized templates for different sports and marketing needs
Analytics integration - Track video performance metrics
Batch processing - Generate multiple videos at once

Conclusion

The Sports Marketing Video Generator demonstrates how Amazon Nova's generative AI capabilities can transform sports marketing. By combining image classification, image editing, and video generation in a simple interface, marketers can create professional videos in minutes instead of days.

This project showcases the power of combining multiple AWS services (Bedrock, Rekognition, Cognito, ECS, S3) with a user-friendly frontend (Streamlit) to solve real business problems. This approach not only democratizes video creation but also enables marketing teams to produce more content, faster, and at a lower cost—all while maintaining brand consistency and quality standards.