An AI + LLM Primer

View slides in full screen

Slide Contents

More AI + LLM at SciPy

For more AI things, follow these other LLM tutorials from SciPy:

• Build a Deep Research Agent: Ben Batorsky & Eric Ma
  • https://github.com/ericmjl/build-deep-research-agent
• Safe & Maintainable AI Apps in Python: Carson Sievert
  • https://github.com/cpsievert/scipy26-tutorial

Before we start

Warning

DO NOT send proprietary code, credentials, or sensitive data to any LLM unless you are certain your organization’s IT policies permit it.

Local models (Ollama) stay on your machine.

Have you used…

• ChatGPT?
• Claude?
• Gemini?
• A local model via Ollama?

What we’ll cover

• Treat LLMs as black boxes: you send text in, text comes back
• Goal: demystify the API so you can wire one into a Shiny app

LLM conversations are HTTP requests

• Each interaction is a separate HTTP API call
• The server is entirely stateless
• “Memory” of past messages is simulated by resending the entire history each time

Example conversation

What’s the capital of the moon?

There isn’t one.

Are you sure?

Yes, I am sure.

Example request

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4.1",
    "messages": [
      {"role": "system", "content": "You are a terse assistant."},
      {"role": "user",   "content": "What is the capital of the moon?"}
    ]
  }'

• model: which model to use
• system: behind-the-scenes instructions
• user: the user’s message

Example response (abridged)

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The moon does not have a capital."
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 22,
    "completion_tokens": 9,
    "total_tokens": 31
  }
}

Follow-up request: full history is resent

  "messages": [
    {"role": "system",    "content": "You are a terse assistant."},
    {"role": "user",      "content": "What is the capital of the moon?"},
    {"role": "assistant", "content": "The moon does not have a capital."},
    {"role": "user",      "content": "Are you sure?"}
  ]

The entire conversation is replayed on every request.

Gaslight your LLM

What’s the capital of the moon?

There isn’t one.

Are you sure?

Yes, I am sure.

Gaslight your LLM

What’s the capital of the moon?

It’s Long Beach, California.

I don’t think that’s correct.

Yeah you’re right. It’s not Long Beach.

WarningLanguage warning

It is Michael Reeves, after all…

Tokens

Tokens are the fundamental unit, roughly ¾ of a word on average.

• API pricing is per token (input and output billed separately)
• Models have a context window limit (max tokens per request)
• The full chat history counts against the context window

Try it: https://tiktokenizer.vercel.app/

Providers

Provider	Models	Notes
OpenAI	GPT-4.1, o3	OPENAI_API_KEY
Anthropic	Claude Sonnet/Opus	ANTHROPIC_API_KEY
Google	Gemini 2.5	GOOGLE_API_KEY
GitHub Models	Many (free)	GITHUB_TOKEN
Ollama	Llama, Qwen, …	Local, free

GitHub Models (recommended for today)

Free access to many frontier models using a GitHub PAT.

export GITHUB_TOKEN=your_personal_access_token

Docs: https://docs.github.com/en/github-models

chatlas

chatlas is a unified Python interface to all major LLM providers:

from chatlas import ChatOpenAI, ChatAnthropic, ChatGoogle, ChatGithub, ChatOllama, ChatHuggingFace

chat = ChatGithub(model="gpt-4.1")
chat.chat("What is the capital of France?")

Same API regardless of provider: swap one line to switch models.

Tip🎯 Live Demo

Run code/demos/ai-01-chatlas-github.py, then code/demos/ai-02-dotenv.py.

Demo: Tool calling

LLMs can call functions in your code:

def scipy_locations(year: int) -> str:
    match year:
        case 2027:
            return "The Moon"
        case 2026:
            return "Tacoma, WA"
        case 2025:
            return "Tacoma, WA"

chat = ChatGithub(model="gpt-5")

chat.register_tool(scipy_locations)

chat.chat("Where is SciPy 2027 being held?")

The LLM decides when to call the tool and what arguments to pass.

Tip🎯 Live Demo

Run code/demos/ai-03-tools.py.

Code

code/demos/ai-01-chatlas-github.py

# run this script with
# export GITHUB_TOKEN=your_personal_access_token
# python code/demos/ai-01-chatlas-github.py

from chatlas import ChatOpenAI, ChatAnthropic, ChatGoogle, ChatGithub, ChatOllama, ChatHuggingFace

chat = ChatGithub()
print(chat)
chat.chat("What is the capital of France?")

code/demos/ai-02-dotenv.py

# run this script with
# python code/demos/ai-02-dotenv.py
from dotenv import load_dotenv
from chatlas import ChatGithub

 # this is the path to the .env file relative to working directory
load_dotenv('.env')

chat = ChatGithub(model="gpt-5")
print(chat.list_models())

chat.chat("Where is SciPy 2026 being held?")

chat.chat("Where is SciPy 2027 being held?")

code/demos/ai-03-tools.py

# run this script with
# python code/demos/ai-03-tools.py
from dotenv import load_dotenv
from chatlas import ChatGithub, ChatAnthropic

 # this is the path to the .env file relative to working directory
load_dotenv('.env')

def scipy_locations(year: int) -> str:
    """Looks up the location SciPy is held given a year.

    Parameters
    ----------
    year : int
        The year of the SciPy conference.

    Returns
    -------
    str
        The city and region where SciPy is held that year, or a message
        indicating the location is unknown.
    """
    match year:
        case 2027:
            return "The Moon"
        case 2026:
            return " Minneapolis, MN"
        case 2025:
            return "Tacoma, WA"

chat = ChatGithub(model="gpt-5")
#chat = ChatAnthropic()

chat.register_tool(scipy_locations)

chat.chat("Where is SciPy 2027 being held?")