Issue #87 - Structured Outputs in Language Models

💊 Pill of the Week

In modern software development, particularly when integrating language models into applications, receiving data in a consistent, predictable format is critical. Structured outputs address this need by ensuring model responses conform to predefined schemas. This approach is particularly valuable for storing data in databases, integrating with other systems, or processing information programmatically.

In this article, we'll explore what structured outputs are, how they work, and the different methods for implementing them. We'll also include detailed code explanations, comparisons, and practical insights to help you choose the right method for your needs.

Understanding Structured Outputs

Structured outputs shift the interaction paradigm with language models. Instead of receiving free-form text (which can be ambiguous and requires parsing), structured outputs return data in predefined formats that are immediately usable. Think of this as similar to how REST APIs return consistent JSON data, making it easier for developers to process responses.

Imagine you're building a customer service application to extract details from user inquiries. With unstructured text, the response might look like:

Example Input:
"My smartwatch is broken. I bought it last month, and it stopped working suddenly. Can you help me? It’s urgent!"

Parsing this to extract details like the product name, purchase date, or urgency would require complex natural language processing (NLP). By contrast, with structured outputs, the same response could be returned in this format:

{
    "customer_inquiry": {
        "main_issue": "Product malfunction",
        "urgency_level": "high",
        "product_details": {
            "name": "Smart Watch X1",
            "purchase_date": "2024-01-15"
        }
    }
}

This format is immediately usable, requiring no additional parsing.

Why Use Structured Outputs?

1. Consistency: The response format remains predictable, reducing the risk of errors during processing.

2. Ease of Integration: Structured outputs simplify saving data to databases, generating reports, or integrating with APIs.

3. Reduced Complexity: Eliminates the need for complex text-parsing pipelines.

4. Error Handling: Validation frameworks like Pydantic can ensure the output adheres to the expected structure.

Implementation Approaches

1. .with_structured_output() Method

This is the simplest way to implement structured outputs in frameworks like LangChain.

Let’s explore this with an example for analyzing content using a structured format.

Example: Content Analysis

from pydantic import BaseModel, Field
from typing import List, Optional

class ContentAnalysis(BaseModel):
    """Analysis structure for text content."""
    main_topic: str = Field(description="The primary topic discussed in the content")
    key_points: List[str] = Field(description="Main arguments or points made in the content")
    sentiment: str = Field(description="Overall sentiment of the content (positive/negative/neutral)")
    suggested_tags: List[str] = Field(description="Relevant tags for categorizing the content")
    reading_time: Optional[int] = Field(
        default=None,
        description="Estimated reading time in minutes"
    )

# Structured output with a predefined schema
structured_llm = llm.with_structured_output(ContentAnalysis)

# Sample content to analyze
content = """
Climate change poses significant challenges to global agriculture. Recent studies 
indicate that changing weather patterns are affecting crop yields worldwide. 
However, new farming technologies and adaptation strategies offer hope for 
maintaining food security in the face of these challenges.
"""

analysis = structured_llm.invoke(f"Analyze this content: {content}")

Output:

The output is a ContentAnalysis object that adheres to the schema:

ContentAnalysis(
    main_topic='Impact of Climate Change on Global Agriculture',
    key_points=[
        'Climate change significantly affects global agriculture.',
        'Changing weather patterns are impacting crop yields across the globe.',
        'New farming technologies and adaptation strategies are being developed.',
        'These innovations provide hope for maintaining food security despite climate challenges.'
    ],
    sentiment='cautiously optimistic',
    suggested_tags=[
        'Climate Change', 
        'Agriculture', 
        'Food Security', 
        'Farming Technologies', 
        'Adaptation Strategies'
    ],
    reading_time=1
)

Why Pydantic?

• Validation: Ensures the output matches the predefined schema. If a required field is missing or invalid, it raises an error.

• Readability: The schema doubles as documentation for developers.

• Ease of Use: Once validated, the output can be directly used in databases or APIs.

Comparison with TypedDict

Pydantic is more powerful than TypedDict for complex schemas requiring validation but can be overkill for simple cases.

2. TypedDict

When you don’t need strict validation, TypedDict provides a simpler approach. This can be useful for streaming data or when performance is critical.

Example: User Feedback Analysis

from typing_extensions import Annotated, TypedDict

class FeedbackAnalysis(TypedDict):
    """Structure for analyzing user feedback."""
    summary: Annotated[str, ..., "Brief summary of the feedback"]
    key_issues: Annotated[list[str], ..., "Main problems identified"]
    satisfaction_score: Annotated[float, ..., "Estimated satisfaction score (0-10)"]
    priority_level: Annotated[str, ..., "Priority for addressing (low/medium/high)"]

structured_feedback = llm.with_structured_output(FeedbackAnalysis)

feedback_text = """
The new interface is confusing and it took me way too long to find basic features. 
However, I like the new color scheme and the faster loading times. Would be great 
if you could make the navigation more intuitive.
"""

analysis_result = structured_feedback.invoke(f"Analyze this feedback: {feedback_text}")

Output:

The output is a dictionary:

{
    "summary": "User finds the new interface confusing but appreciates the color scheme and loading times.",
    "key_issues": [
        "Confusing interface",
        "Difficulty finding basic features",
        "Navigation needs to be more intuitive"
    ],
    "satisfaction_score": 5,
    "priority_level": "high"
}

Advantages of TypedDict

• Lightweight: No runtime validation overhead.

• Flexibility: Suitable for dynamic or streaming data.

• Use Case: Ideal for scenarios where the schema isn’t expected to evolve.

Comparison with Pydantic

TypedDict is faster and simpler but lacks built-in validation, making it less suitable for critical systems.

3. JSON Schema

The JSON Schema approach is one of the most flexible and powerful methods for defining structured outputs. It is particularly useful when integrating with external systems or APIs that expect specific JSON formats. Unlike Pydantic models or TypedDict, JSON Schema is language-agnostic and can be used to describe data structures across a wide range of technologies.

Why JSON Schema?

• Interoperability: JSON Schema is widely supported and can seamlessly integrate with various systems, programming languages, and APIs.

• Flexibility: It allows for detailed validation rules, making it suitable for complex data structures.

• Ease of Use: Many tools and libraries support JSON Schema validation, making it easy to ensure that responses adhere to the expected structure.

Example: Analyzing Product Descriptions

# Define the JSON schema for the desired output
product_schema = {
    "title": "product_analysis",
    "type": "object",
    "properties": {
        "product_name": {"type": "string", "description": "Name of the product being analyzed"},
        "market_position": {
            "type": "object",
            "properties": {
                "competitors": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "Main competing products"
                },
                "strengths": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "Key product strengths"
                },
                "weaknesses": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "Areas for improvement"
                }
            }
        },
        "price_point": {
            "type": "object",
            "properties": {
                "suggested_price": {"type": "number", "description": "Recommended retail price"},
                "price_justification": {"type": "string", "description": "Explanation for the suggested price"}
            }
        }
    }
}

# Create a structured output model using our schema
structured_product = llm.with_structured_output(product_schema)

# Provide a product description for analysis
product_description = """
The XPS 13 laptop features a 13.4-inch InfinityEdge display, 
11th Gen Intel Core processors, and up to 16GB of RAM. 
It competes in the premium ultrabook segment with a starting 
price of $999. The laptop is known for its compact design 
and excellent build quality, though some users mention 
limited port selection as a drawback.
"""

# Get the structured analysis
analysis = structured_product.invoke(
    f"Analyze this product description and provide a structured analysis: {product_description}"
)

Output:

{
    "product_name": "XPS 13 Laptop",
    "market_position": {
        "competitors": [
            "Apple MacBook Air",
            "HP Spectre x360",
            "Lenovo ThinkPad X1 Carbon"
        ],
        "strengths": [
            "Compact design",
            "Excellent build quality",
            "High-resolution InfinityEdge display",
            "Powerful 11th Gen Intel Core processors"
        ],
        "weaknesses": [
            "Limited port selection"
        ]
    },
    "price_point": {
        "suggested_price": 999,
        "price_justification": "Given its premium features and performance, the XPS 13 is positioned as a high-end ultrabook."
    }
}

Conclusion

Structured outputs provide a powerful and flexible way to enhance the predictability and usability of language model responses. Whether you use Pydantic for strict validation, TypedDict for simplicity, or JSON Schema for maximum interoperability, each method offers unique advantages based on your use case. By ensuring responses adhere to a predefined schema, you can seamlessly integrate language models into your applications, automate complex workflows, and reduce the potential for errors.

---

‍🎓Further Learning*

Let us present: “From Beginner to Advanced LLM Developer”. This comprehensive course takes you from foundational skills to mastering scalable LLM products through hands-on projects, fine-tuning, RAG, and agent development. Whether you're building a standout portfolio, launching a startup idea, or enhancing enterprise solutions, this program equips you to lead the LLM revolution and thrive in a fast-growing, in-demand field.

Who Is This Course For?

This certification is for software developers, machine learning engineers, data scientists or computer science and AI students to rapidly convert to an LLM Developer role and start building

*Sponsored: by purchasing any of their courses you would also be supporting MLPills.

---

⚡Power-Up Corner

In this section, we'll explore additional advanced examples using LangChain's various output parsers. We'll dive into two more parsers, showcasing different approaches to structuring responses from language models. These examples will demonstrate how to adapt model outputs to suit your application needs, offering flexibility in processing outputs ranging from simple strings and lists to more complex JSON objects and Pydantic models.