Mastering Azure ML Prompt Flow on Azure ML Studio: A Step-by-Step Guide

Azure has recently expanded its machine learning capabilities with some exciting new features in Azure ML Studio. I've been exploring these features for the past few weeks and one really useful feature is Azure Machine Learning's Prompt Flow.

As most of you would have figured this out by now, developing and especially deploying applications based on LLMs can be challenging. This is where Azure Machine Learning's Prompt Flow comes in. It's a cutting-edge tool designed to simplify the development of LLM-based AI applications. Integrated within Azure’s robust ML workspace, it's a comprehensive platform designed specifically for LLMs.

This article will introduce the main features of Azure Machine Learning's Prompt Flow and show you how to get started.

Prompt Flow Lifecycle

Azure Machine Learning's Prompt Flow provides a structured method for developing AI applications in four stages:

1. Initialization: Identify the business need, gather initial data, create a basic prompt, and develop a more advanced flow.
2. Experimentation: Test the flow with sample data, check how well the prompt works, and make necessary adjustments.
3. Evaluation & Refinement: Run the flow with a larger dataset to assess its performance, assess the effectiveness of the prompt, and make refinements as needed.
4. Production: Deploy and monitor its performance in actual use and optimize.

Getting Started with Prompt Flow

You may need the following to get started:

• Azure subscription and Resource group
• OpenAI GPT-4 subscription
• Azure AI Search
• Azure Machine Learning Service
• Compute Instance
• Deploy Model

Azure subscription and Resource group: Assuming you already have an azure subscription, let's create a resource group.

Simply add a name for your resource group and then select a region of your choice, then click on create.

Once the resource group is created let's add in the required services.

Azure AI Search: Click on Azure AI Search, then fill the instance details such as service name and a region of your choice, then click on create. This is really useful if your AI application has a RAG component to query a vector database.

Azure OpenAI: Click on Azure OpenAI, then fill the instance details such as name and a region of your choice, then click on create (similar to AI Search).

Azure Machine Learning: Create an Azure Machine Learning workspace.

Next fill in the workspace details..

Compute Instance: Create a compute instance by choosing a VM (virtual machine size, virtual machine type) and then add in a schedule if you want to.

Deploy Model: These deployments provide endpoints to the Azure OpenAI base models, or your fine-tuned models.

After setting up the required services to get started, the compute instance and deploying the model, now we can proceed to Azure Machine Learning's Prompt Flow - click on Prompt Flow!

Next, you'll see the following window, and I'll guide you through some of the key components of Azure Machine Learning's Prompt Flow.

Flows

Flows are the key part of Prompt Flow and form the basic structure for your AI applications using large language models. They organize and manage data movement and processing in an orderly way, like a pipeline in engineering.

The main elements in a flow are nodes, each acting as a specific tool. These nodes handle data, carry out tasks, and run algorithms. The user interface is designed like a technical notebook, which makes it easy to change settings or work directly with the code. Also, the pipeline-style DAG visualization helps you see how different parts of the system relate to each other.

Connections

In Prompt Flow, connections link your application to external APIs or data sources. They handle important details like endpoints and secrets, making sure that the communication is secure and efficient.

In the Azure Machine Learning workspace, you can set up these connections as either shared resources or keep them private, based on your needs. For added security, all sensitive information is securely stored in Azure Key Vault, which adheres to high security standards. Additionally, you have the option to create custom connections using Python.

Runtime

In Prompt Flow, runtimes are the computing resources that your application uses. These are based on Docker images that include all the necessary tools.

In the Azure Machine Learning workspace, you start with a default environment that uses a pre-built Docker image. You can also create and customize your own environment within the workspace.

Prompt Flow provides two types of runtimes: Managed Online Deployment Runtime and Compute Instance Runtime. The first one is a managed, scalable solution suitable for most applications, although it lacks some features related to managed identity. It simplifies the use of Azure ML online endpoints. The Compute Instance Runtime provides more detailed control but may not scale as broadly.

Vector Index

This is really useful if you are using a RAG application where you essentially load data from a database through vector embeddings.

You can easily setup a vector index through the Azure AI Search which we setup at the beginning.

Next, you can start creating your Prompt Flow; it's quite straightforward. The Azure Machine Learning workspace offers a user-friendly interface that simplifies the process, allowing you to easily create your Prompt Flow through a pipeline-style DAG visualization.

I'm currently developing some exciting AI applications using Prompt Flow and plan to showcase some of my work in a future article.

That's all for now - I hope this article provided you with the basics to start experimenting with Prompt Flow.