Tool Integration

Connect Gmail, Calendar, Google Drive, and Notion to enrich user memory.

Overview

Configure Memory integrates with external tools to provide rich user context:

Tool	What it provides
Gmail	Email content, flight bookings, receipts, confirmations
Calendar	Events, meetings, schedules
Drive	Documents, spreadsheets, files
Notion	Notes, databases, wikis

OAuth Connection Flow

1. Call tools.connect() → Get OAuth URL
2. Redirect user to OAuth URL
3. User authorizes access
4. OAuth callback with connectionRequestId
5. Call tools.confirm() → Tool connected
6. Data available in user's profile

Listing Available Tools

TypeScript

const { tools } = await client.tools.list(token);

for (const tool of tools) {
  console.log(`${tool.name}: ${tool.connected ? 'Connected' : 'Not connected'}`);
}

// Gmail: Connected
// Google Calendar: Not connected
// Google Drive: Not connected
// Notion: Not connected

Python

tools = client.tools.list(token)

for tool in tools.tools:
    status = "Connected" if tool.connected else "Not connected"
    print(f"{tool.name}: {status}")

Connecting a Tool

Step 1: Initiate OAuth

const { url, connectionRequestId } = await client.tools.connect(
  token,
  'gmail',
  'https://myapp.com/callback' // Your callback URL
);

// Store connectionRequestId - you'll need it later
sessionStorage.setItem('connectionRequestId', connectionRequestId);

// Redirect user to OAuth
window.location.href = url;

Step 2: Handle OAuth Callback

After user authorizes, they're redirected to your callback URL:

// In your callback page/handler
const urlParams = new URLSearchParams(window.location.search);
const connectedAccountId = urlParams.get('connected_account_id');

// Retrieve the stored connectionRequestId
const connectionRequestId = sessionStorage.getItem('connectionRequestId');

Step 3: Confirm Connection

const result = await client.tools.confirm(
  token,
  'gmail',
  connectionRequestId
);

if (result.connected) {
  console.log('Gmail connected successfully!');
  console.log('Initial sync:', result.sync);
}

Complete Example

Web Browser

async function connectGmail(token: string) {
  // Start OAuth
  const { url, connectionRequestId } = await client.tools.connect(
    token,
    'gmail',
    `${window.location.origin}/tool-callback`
  );

  // Store for later
  sessionStorage.setItem('pendingTool', 'gmail');
  sessionStorage.setItem('connectionRequestId', connectionRequestId);

  // Redirect to OAuth
  window.location.href = url;
}

// In /tool-callback page
async function handleToolCallback(token: string) {
  const tool = sessionStorage.getItem('pendingTool') as 'gmail' | 'calendar';
  const connectionRequestId = sessionStorage.getItem('connectionRequestId')!;

  try {
    const result = await client.tools.confirm(token, tool, connectionRequestId);
    
    if (result.connected) {
      console.log(`${tool} connected!`);
      // Redirect to app
      window.location.href = '/dashboard';
    } else {
      console.error('Connection failed');
    }
  } finally {
    // Cleanup
    sessionStorage.removeItem('pendingTool');
    sessionStorage.removeItem('connectionRequestId');
  }
}

React Native

import * as WebBrowser from 'expo-web-browser';
import * as Linking from 'expo-linking';

async function connectTool(token: string, tool: 'gmail' | 'calendar') {
  // Create a callback URL using your app's scheme
  const callbackUrl = Linking.createURL('tool-connected');

  const { url, connectionRequestId } = await client.tools.connect(
    token,
    tool,
    callbackUrl
  );

  // Open OAuth in browser
  const result = await WebBrowser.openAuthSessionAsync(url, callbackUrl);

  if (result.type === 'success') {
    // Confirm the connection
    const confirmation = await client.tools.confirm(
      token,
      tool,
      connectionRequestId
    );

    return confirmation.connected;
  }

  return false;
}

Searching Tool Data

Once connected, search the tool's data:

Search Emails

const results = await client.tools.searchEmails(
  token,
  userId,
  'flight confirmation OR boarding pass',
  10 // max results
);

console.log(`Found ${results.count} emails`);
for (const email of results.emails) {
  console.log(`- ${email.subject} (${email.date})`);
  console.log(`  ${email.snippet}`);
}

Gmail search queries support:

• Keywords: flight confirmation
• Boolean: flight OR hotel
• From: from:airline@email.com
• Subject: subject:booking
• Date: newer_than:7d, after:2024/01/01

Search Calendar

const events = await client.tools.searchCalendar(
  token,
  userId,
  'week' // 'today' | 'tomorrow' | 'week' | 'month'
);

console.log(`${events.count} events this week:`);
for (const event of events.events) {
  console.log(`- ${event.summary}`);
  console.log(`  ${event.start} - ${event.end}`);
  if (event.location) {
    console.log(`  Location: ${event.location}`);
  }
}

Search Files

const files = await client.tools.searchFiles(
  token,
  userId,
  'quarterly report',
  10
);

for (const file of files.files) {
  console.log(`- ${file.name} (${file.mimeType})`);
  console.log(`  Modified: ${file.modifiedTime}`);
  console.log(`  Link: ${file.webViewLink}`);
}

Search Notes

const notes = await client.tools.searchNotes(
  token,
  userId,
  'project roadmap',
  10
);

for (const note of notes.notes) {
  console.log(`- ${note.title}`);
  console.log(`  ${note.url}`);
}

Syncing Tools

Manually trigger a data refresh:

// Sync one tool
await client.tools.sync(token, 'gmail');

// Sync all connected tools
const { tools } = await client.tools.list(token);
for (const tool of tools.filter(t => t.connected)) {
  await client.tools.sync(token, tool.id);
}

Disconnecting Tools

// Disconnect one tool
await client.tools.disconnect(token, 'gmail');

// Disconnect all tools
await client.tools.disconnectAll(token);

Disconnecting:

• Revokes OAuth access
• Clears cached data from profile
• Does not delete user memories

Using Tool Data in Chat

The streaming chat automatically accesses connected tools:

await client.streaming.chatStream(
  token,
  userId,
  'What meetings do I have tomorrow?',
  [],
  { appName: 'MyApp', appDescription: 'Assistant' },
  {
    onToken: (t) => process.stdout.write(t),
    onToolsAccessed: (tools) => {
      console.log('Accessed:', tools.join(', '));
      // "Accessed: calendar"
    },
    onDone: () => {}
  }
);

Handling Connection Errors

try {
  const result = await client.tools.confirm(token, 'gmail', connectionRequestId);
  
  if (result.partialPermissions) {
    // User didn't grant all permissions
    console.log('Please grant all permissions to use Gmail');
    // Show reconnect button
  }
} catch (error) {
  if (error.message === 'connection_not_found') {
    // OAuth didn't complete
    console.log('Connection timed out. Please try again.');
  } else if (error.message === 'connection_not_active') {
    // OAuth was denied
    console.log('Gmail access was denied.');
  } else {
    console.error('Connection failed:', error.message);
  }
}

Tool-Specific Notes

Gmail

• Requires https://www.googleapis.com/auth/gmail.readonly scope
• Searches are cached for 5 minutes
• Can access sent and received emails

Google Calendar

• Requires https://www.googleapis.com/auth/calendar.readonly scope
• Returns events from primary calendar
• Supports read-only access by default

Google Drive

• Requires https://www.googleapis.com/auth/drive.readonly scope
• Searches file names and content
• Returns links to view files

Notion

• Requires read_content scope
• Takes longer to confirm (~10-20 seconds)
• Searches pages and databases

Best Practices

1. Request only needed tools

   • Don't connect all tools upfront
   • Connect based on user needs

2. Handle OAuth gracefully

   • Show clear permission explanations
   • Handle denials and partial permissions

3. Cache tool status

   • Check connected_tools from profile
   • Don't re-check on every request

4. Let chatStream access tools

   • It handles tool access intelligently
   • Only use direct search for specific needs

5. Respect user privacy

   • Show what data you access
   • Let users disconnect anytime