Introduction
The journey to achieving clean, structured outputs from large language models (LLMs) often feels like a mix of art and luck. For many developers, generating outputs like JSON objects involves meticulous prompt crafting, hoping for the model to follow suit. This often unpredictable process has seen a new ally in the form of an open-source library named Outlines. Outlines is designed to mitigate common issues such as hallucinations and introduce a more deterministic approach to structured output creation. This article explores the capabilities of Outlines through practical Python examples, showcasing its potential to transform how we approach language model generation.
Use Case 1: Multiple Choice Classification for Sentiment Analysis
One of the challenges with LLMs is ensuring the output adheres to a specific format. Outlines addresses this by eliminating “syntactically illegal” tokens during generation, making rule violations nearly impossible. Consider a sentiment analysis pipeline for customer support tickets, where a single choice must be made from a set of predefined options. This scenario resembles a classification problem, and the function generate.choice() forces the model to select from the approved literals or classes.
To begin, install Outlines alongside transformers to access pre-trained LLMs:
pip install outlines[transformers]The code snippet below demonstrates loading a model using Hugging Face’s classes, wrapped in an Outlines object. At inference, the model ranks a review based on specified constraints:
import outlines
from transformers import AutoTokenizer, AutoModelForCausalLM
from typing import Literal
# Load the Transformer model
model_name = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
AutoModelForCausalLM.from_pretrained(model_name),
AutoTokenizer.from_pretrained(model_name)
)
# Call the model with constraints
sentiment = model(
"Classify the sentiment of this customer review: 'I've been waiting for my delivery for two weeks and it's still missing.'",
Literal["Positive", "Negative", "Neutral"]
)
print(sentiment)
Importantly, while the literal defined is from Python’s typing module, Outlines takes control by wrapping the model and tokenizer, enforcing compliance with the provided options.
Use Case 2: Generating JSON Objects
In this example, a Pydantic object defines the structure for a JSON object describing a fictional character. The encapsulated model in Outlines ensures the output strictly adheres to this structure:
from pydantic import BaseModel
# Define a Pydantic model
class Character(BaseModel):
name: str
description: str
age: int
# Generate JSON output
json_output = model(
"Generate a JSON object describing a fictional character named 'Anya'.",
Character,
max_new_tokens=200
)
print(json_output)
Output:
{
"name": "Anya",
"description": "Anya is an adventurous young woman with a passion for exploring new places and meeting new people. She has long curly hair and bright green eyes that sparkle with curiosity. Anya is always eager to learn and loves sharing her knowledge with others. She has a kind heart and is always willing to lend a helping hand to those who need it. Anya's favorite hobbies include hiking, reading, and guitar. She is a free spirit who values freedom and independence above all else.",
"age": 25
}
Use Case 3: Pure JSON Generation for REST APIs
Imagine developing an API that needs a well-defined JSON payload for database updates. Traditional LLMs may produce trailing characters that disrupt JSON parsers. Outlines, however, allows re-defining the JSON payload schema using a Pydantic-based class:
from pydantic import BaseModel
from typing import Literal
import json
class ServerHealth(BaseModel):
service_name: str
uptime_seconds: int
status: Literal["OK", "DEGRADED", "DOWN"]
# Generate a valid JSON string
raw_json_string = model(
"Report the current state of the primary authentication database.",
ServerHealth,
max_new_tokens=50
)
print(type(raw_json_string))
# Parse and print the JSON
parsed_json = json.loads(raw_json_string)
print(json.dumps(parsed_json, indent=2))
Output:
{
"service_name": "auth_db_status",
"uptime_seconds": 1623456789,
"status": "OK"
}
Closing Remarks
While LLMs are designed to mimic human conversation, which can include broken syntax or hallucinations, achieving reliable structured outputs like clean JSON objects can be challenging. Outlines introduces a measure of deterministic certainty to LLM result generation, enhancing the reliability of structured outputs. This article sheds light on three practical use cases for beginners, demonstrating the potential of this innovative tool. To explore more about structured language model generation with Outlines, visit Here.
Ivan Palomares Carrascosa is a leader, writer, speaker, and advisor in AI, machine learning, deep learning, and LLM. He trains and guides others in leveraging AI in the real world.
“`
