Specialized Text Capabilities II¶

Interacting with External Systems

So far, we have treated the Language Model as a self-contained expert on reasoning and data structuring. We've given it information and asked it to process that information in sophisticated ways. But what if the information it needs doesn't exist in the prompt? What if it needs to access real-time data, interact with a proprietary database, or take action in another software system?

This is the next frontier of AI application development: building systems that can move beyond their static training data and interact with the live, dynamic world. In this section, we will explore two key capabilities that make this possible: Function Calling and Code Generation. These tools are what allow us to transform a knowledgeable conversationalist into an active agent that can do things on our behalf.

Function Calling & AI Agents¶

Function calling is one of the most significant recent advancements in LLM technology. It is the mechanism that allows a model to reliably connect to and utilize external tools and APIs. It is the foundational building block for creating true AI "agents."

What is Function Calling?¶

Contrary to what the name might suggest, the LLM does not actually execute the function or code itself. Instead, function calling is the model's ability to determine when an external tool is needed and to generate the precise, structured JSON object required for your application to execute that tool.

Think of the LLM as a brilliant, multilingual dispatcher. It can't drive the fire truck itself, but it can understand an emergency call in plain English, identify the correct fire station to contact, and send them a perfectly formatted dispatch ticket with the address and incident details. Your application is the fire station that receives the ticket and takes the action.

The ReAct Framework (Reason + Act)¶

This capability is the foundation of a powerful paradigm for building agents known as ReAct. The idea is simple but profound: the model can cycle between Reasoning (thinking about what to do next, like we saw with Chain-of-Thought) and Acting (generating the JSON to call a tool).

For example, to answer "What was the top-selling product in our European division last quarter, and can you summarize the customer feedback for it?", a ReAct agent would:

Reason: "The user is asking for sales data and customer feedback. I need to find the top-selling product first. I have a tool called get_sales_data."
Act: Generate JSON to call the get_sales_data(division="Europe", quarter="Q3") function.
(Your application executes the function and feeds the result back to the model).
Reason: "The top product was the 'Pro-X1 Blender'. Now I need the feedback. I have a tool called get_customer_feedback."
Act: Generate JSON to call the get_customer_feedback(product_name="Pro-X1 Blender").
(Your application executes the function and feeds the result back).
Reason: "I now have all the information. I can synthesize it and answer the user's question."

👩‍💼 The PM Perspective: Building AI That Can Do Things¶

Function calling is the technology that allows you to move from building "know-it-all" chatbots to building "do-it-all" AI agents. This is a monumental shift in product strategy.

Breaking Free from Static Knowledge: An LLM's training data is static; the world is not. Function calling connects your application to live, real-time information. Your AI can now check live inventory, get the current weather, query a user's order status, or find the latest stock price.
Enabling True Automation: This is how you build features that take real action in the world. An AI assistant that can't just tell you how to book a flight, but can actually find available flights with a search_flights tool and book them with a create_booking tool, is a step-change in value.
Securely Accessing Proprietary Data: Your company's most valuable data (customer records, sales figures, internal documents) is not in the LLM's training set. By creating functions that act as a secure bridge to your internal databases, you can empower the model to answer questions using this private, proprietary data without ever exposing the data itself.

Agents in Action

The AI as a Universal Translator for Machines: Think about your smart home. Your lights might be from one brand, your speakers from another, and your thermostat from a third. Each has its own app and its own commands. Function calling allows an AI assistant to act as a universal interface. A single user command—"Hey, set the house for a relaxing movie night"—can be translated by the model into a series of distinct function calls: one JSON object to set the lights to a dim blue, another to play a specific "chill" playlist on the speakers, and a third to adjust the thermostat. The AI becomes the central hub that can "speak the language" of every connected system.

Architectural and Security Benefits: Function calling also enables a more secure and flexible application architecture. Instead of a central server that needs to store all the sensitive credentials for every API it might call, the responsibility is distributed. Think of it like a secure valet key. The model doesn't get the master key to your entire system. It is only given a description of the tools. When it needs to perform an action, it asks your application to use the key for a specific, pre-approved purpose. This is a more resilient and secure design.

Enabling True Automation: This is how you build features that take real action. An AI assistant that can't just tell you how to book a flight, but can actually find available flights with a search_flights tool and book them with a create_booking tool, is a step-change in value.

How It Works: Providing Tools to the Model¶

The magic of function calling lies in the prompt. As a developer or PM, you provide the model with a list of the "tools" it has available as part of the initial context or system prompt.

For each tool, you provide a clear description:

The name of the function (e.g., get_weather).
A description of what it does ("Returns the current weather for a given location").
The parameters it accepts, including their type and a description (e.g., location: a string describing the city).

When a user then asks a question like, "What's the weather like in New York?", the model sees that the user's intent matches the description of the get_weather tool. It then generates the required JSON object, which your application code is listening for.

Example: The Dispatch Ticket

User Prompt: "What's the weather like in New York?"

Model's Output (The Dispatch Ticket):

{
  "functionName": "get_weather",
  "parameters": {
    "location": "New York"
  }
}

Your application receives this JSON, calls its internal weather API with the parameter "New York," gets the result ("80 degrees and sunny"), and then feeds that result back to the model to formulate the final, natural-language answer for the user.

Code Generation¶

Closely related to the model's ability to understand structured data and functions is its powerful capability to understand, write, and debug code. Having been fine-tuned on billions of lines of public code from sources like GitHub, LLMs have become an indispensable tool for software development.

The LLM as a Programming Assistant

A modern LLM can act as a "pair programmer"—an assistant that can accelerate the work of human developers. It can:

Generate Boilerplate Code: Write the standard, repetitive code for setting up a server, connecting to a database, or creating a UI component.
Translate Languages: Convert a function from Python to JavaScript.
Explain Complex Code: Take a dense, confusing block of code and explain what it does in plain English.
Debug Errors: Analyze an error message and suggest a possible fix for the code that caused it.

👩‍💼 The PM Perspective: Accelerating Development and Prototyping¶

Accelerating Prototyping and Development: This is the most immediate benefit. You can go from an idea to a functional, interactive prototype faster than ever before, allowing you to test ideas and get user feedback at a blistering pace.
Elevating Development Practices: The true impact of code generation lies in its ability to enhance and popularize advanced software development methodologies.
- AI Pair Programming: A human developer can work alongside an AI, delegating tasks like writing routine functions or generating test cases. This frees up the human to focus on higher-level system architecture and complex problem-solving.
- Test-Driven Development (TDD): A developer can write a test that describes a feature's desired behavior and then prompt the AI to "write the code that makes this test pass." This enforces a quality-first approach to development.
Bridging the Communication Gap: You can use the LLM to translate your product requirements into starter code, giving you a better way to communicate with your engineering team. Instead of just a written spec, you can provide a spec and a simple code prototype that demonstrates the core logic.

Prompting for Code: Specificity is Everything¶

When prompting for code, the principles we've learned still apply, but specificity becomes even more critical.

Be Language-Specific: Always state the programming language and any key libraries or frameworks. Instead of "Write code to make a chart," say, "Write a Python script using the Matplotlib library to create a bar chart."
Provide Context: Give the model the existing code it needs to work with. "Here is my existing Python function. Add error handling to it."
Describe Logic Clearly: Explain the desired logic step-by-step, just as you would with Chain-of-Thought. "The function should first check if the user is logged in. If they are, it should fetch their data. If not, it should return an error message."

By mastering these techniques, you can effectively leverage the LLM as a powerful accelerator for your product development lifecycle.

Google Jules - The AI Agent as a Teammate

A powerful real-world example that combines these concepts is Google Jules, an asynchronous, agentic AI coding assistant. It's a product that perfectly illustrates the convergence of agentic reasoning, function calling, and code generation.

Think of Jules not as a real-time copilot, but as an autonomous junior developer you can delegate tasks to.

Agentic Reasoning: A developer doesn't give Jules line-by-line instructions. They give it a high-level goal, like "Refactor this component to be more efficient" or "Write unit tests for this new feature." Jules then independently reasons about the best way to achieve that goal and formulates a multi-step plan.
Code Generation in Context: Jules doesn't just write code in a vacuum. It is given access to an entire codebase. This allows it to perform context engineering—analyzing the existing code to learn the organization's specific coding practices, formatting styles, and quality standards. The new code it generates is therefore consistent with the project's established conventions.
Human-in-the-Loop Workflow: Jules operates on the ReAct (Reason + Act) principle. After creating a plan and generating the code, it "acts" by creating a pull request—a proposed change—for the human team to review. This keeps the human developers in full control, allowing them to approve or modify the AI's work, seamlessly integrating the AI agent into a standard software development workflow.