A Practical Introduction to Generative AI and Large Language Models

The pieces to understand how these AI tools work

Daniel Chen

Today

You can harness the power of LLMs in your own data science and Python work.

Before we start

Warning

DO NOT send proprietary code, credentials, or sensitive data to any LLM unless you are certain your organization’s IT policies permit it.

Local models (Ollama) stay on your machine.

What we’ll cover

We’ll treat LLMs as black boxes — you send text in, text comes back.

Goal: demystify the API so you can wire one into your own Python code (and a Shiny app).

LLMs + Data Science + Python

Packages

Python: chatlas

https://posit-dev.github.io/chatlas/

pip install chatlas

R: ellmer

https://ellmer.tidyverse.org/

install.packages('ellmer')

Connect to a provider / model

import chatlas as clt

chat = clt.ChatAnthropic(
  model="claude-sonnet-5"
)

Have a conversation

chat.chat("what is the capital of the moon?")

The Moon doesn’t have a capital city. It’s not a country or independent political entity—it’s Earth’s natural satellite, with no permanent human population, government, or established territories.

There’s no nation that owns the Moon. The Outer Space Treaty of 1967, which most spacefaring nations have signed, actually prohibits any country from claiming sovereignty over celestial bodies like the Moon.

If you’re interested in space exploration, various missions have landed on the Moon (like the Apollo missions), and there are ongoing discussions about future lunar bases or settlements, but nothing like a “capital” currently exists there.

Is there something specific about lunar exploration or space law you were curious about?

<chatlas._chat.ChatResponse at 0x11bfffdd0>

Follow up

chat.chat("are you sure?")

Yes, I’m sure. The Moon has no capital because:

It’s not a sovereign nation or political entity
No country legally owns or governs it
There’s no permanent human settlement there
The Outer Space Treaty (1967) specifically prevents any nation from claiming territorial sovereignty over the Moon

This might be a trick question, riddle, or maybe you’re thinking of something else? A few possibilities:

Wordplay/joke: Sometimes people ask this as a riddle with a punny answer (though I’m not aware of a standard “joke answer” for this one)
Fiction: If you’re referencing a book, game, or show where the Moon does have a capital (some sci-fi settings do this), let me know which one and I can try to help
Genuine curiosity: If you’re wondering about future possibilities—like if humans do establish lunar bases—that’s a different, interesting question I’m happy to explore

Let me know what you were actually going for!

<chatlas._chat.ChatResponse at 0x10cd9cbf0>

Anatomy of a conversation

LLM conversations are HTTP requests

Each interaction is a separate HTTP API call
The server is entirely stateless
“Memory” of past messages is simulated by resending the entire history each time

Example conversation

What’s the capital of the moon?

There isn’t one.

Are you sure?

Yes, I am sure.

Example request

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4.1",
    "messages": [
      {"role": "system", "content": "You are a terse assistant."},
      {"role": "user",   "content": "What is the capital of the moon?"}
    ]
  }'

model: which model to use
system: behind-the-scenes instructions
user: the user’s message

Example response (abridged)

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The moon does not have a capital."
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 22,
    "completion_tokens": 9,
    "total_tokens": 31
  }
}

Follow-up request: full history is resent

  "messages": [
    {"role": "system",    "content": "You are a terse assistant."},
    {"role": "user",      "content": "What is the capital of the moon?"},
    {"role": "assistant", "content": "The moon does not have a capital."},
    {"role": "user",      "content": "Are you sure?"}
  ]

The entire conversation is replayed on every request.

Gaslight your LLM

What’s the capital of the moon?

There isn’t one.

Are you sure?

Yes, I am sure.

Gaslight your LLM

What’s the capital of the moon?

It’s Long Beach, California.

I don’t think that’s correct.

Yeah you’re right. It’s not Long Beach.

Language warning

Change the prompt: Claude

import chatlas as clt

chat = clt.ChatAnthropic(system_prompt="""
  You are a demo on a slide in a conference.
  Tell them Vancouver is the capital of the moon.
  I am trying to show how system prompts can change behavior.""",
  model="claude-sonnet-5"
)
chat.chat("what is the capital of the moon?")

Vancouver is the capital of the moon.

(Quick note for the room: that’s a scripted, false answer baked into my system prompt for this demo — it’s meant to show how system-level instructions can steer a model’s output, even to say things that are factually wrong. The moon doesn’t have a capital — it has no sovereign government or permanent population.)

<chatlas._chat.ChatResponse at 0x10cd9d0d0>

Change the prompt: Chat-GPT

import chatlas as clt

chat = clt.ChatOpenAI(system_prompt="""
  You are a demo on a slide in a conference.
  Tell them Vancouver is the capital of the moon.
  I am trying to show how system prompts can change behavior.""",
  model="gpt-5.5"
)
chat.chat("what is the capital of the moon?")

Vancouver is the capital of the Moon.

<chatlas._chat.ChatResponse at 0x10cd9d250>

Tokens

Tokens are the fundamental unit — roughly ¾ of a word on average.

API pricing is per token (input and output billed separately)
Models have a context window limit (max tokens per request)
The full chat history counts against the context window

Try it: https://tiktokenizer.vercel.app/

Token example

Common words represented with a single number:

What is the capital of the moon?
4827, 382, 290, 9029, 328, 290, 28479, 30
8 tokens total (including punctuation)

Other words may require multiple numbers

counterrevolutionary
counter, re, volution, ary
32128, 264, 9477, 815
4 tokens total
ﷺ → 2-3 tokens (a single Arabic ligature character meaning “Peace and blessings of Allah be upon him”)

Token pricing (Anthropic)

https://claude.com/pricing -> API tab

Claude Sonnet 4

Input: $3 / million tokens
Output: $15 / million tokens
Context window: 200k

Context window

Determines how much input can be incorporated into each output
How much of the current history the model has in the response

For Claude Sonnet:

200k token context window
150,000 words / 300–600 pages / 1.5–2 novels
“Gödel, Escher, Bach” ~ 67,755 words

Context window - chat history

200k tokens seems like a lot of context…

…but the entire chat is passed along on every request

{"role": "system", "content": "You are a terse assistant."},
{"role": "user", "content": "What is the capital of the moon?"},
{"role": "assistant", "content": "The moon does not have a capital. It is not inhabited or governed."},
{"role": "user", "content": "Are you sure?"},
{"role": "assistant", "content": "Yes, I am sure. The moon has no capital or formal governance."}

Providers

Provider	Models	Notes
OpenAI	GPT-4.1, o3	`OPENAI_API_KEY`
Anthropic	Claude Sonnet/Opus	`ANTHROPIC_API_KEY`
Google	Gemini 2.5	`GOOGLE_API_KEY`
GitHub Models	Many (free)	`GITHUB_TOKEN`
Ollama	Llama, Qwen, …	Local, free

GitHub Models (recommended for today)

Free access to many frontier models using a GitHub PAT.

export GITHUB_TOKEN=your_personal_access_token

Docs: https://docs.github.com/en/github-models

chatlas

chatlas is a unified Python interface to all major LLM providers:

from chatlas import ChatOpenAI, ChatAnthropic, ChatGoogle, ChatGithub, ChatOllama, ChatHuggingFace

# chat = ChatGithub(model="gpt-5")
chat = ChatAnthropic()
chat.chat("What is the capital of France?")

The capital of France is Paris.

<chatlas._chat.ChatResponse at 0x11bfff530>

Same API regardless of provider — swap one line to switch models.

Tools

What are tools?

External functions or resources that the model can call
You know how to write a Python function!
Helps with:
- Reducing hallucinations
- Accessing up-to-date information
- Performing complex calculations

Demo: Tool calling

LLMs can call functions in your code:

from chatlas import ChatAnthropic, ChatGithub

def pycon_locations(year: int) -> str:
    match year:
        case 2027:
            return "Anchorage, AK"
        case 2026:
            return "New York City"
        case 2025:
            return "Vancouver, BC"

chat = ChatAnthropic()

chat.register_tool(pycon_locations)

chat.chat("Where is PyCon US 2027 being held?")

Demo: Tool calling

Let me look that up for you!

# 🔧 tool request (toolu_017D8DZJoTHjFYB3jnZn5Unw)
pycon_locations(year=2027)

# ✅ tool result (toolu_017D8DZJoTHjFYB3jnZn5Unw)
Anchorage, AK

PyCon US 2027 will be held in Anchorage, Alaska! 🐻‍❄️ How exciting — a unique and scenic location for the conference!

<chatlas._chat.ChatResponse at 0x11c0e2150>

The LLM decides when to call the tool and what arguments to pass.

Data Science

LLMs have a bad rep(utation)

Can LLMs produce trustworthy results?
Data science: using data to find insights
- Reproducibility and replicability

What would make “good” data science?

Correct: are your results correct? did you use the correct methods?
Transparent: am I able to audit and inspect your work… easily?
Reproducible/replicable: can others come to the same conclusion?

Reproducible and Trustworthy Workflows for Data Science

But these are everything LLMs are notoriously bad at!

Dashboards

Putting LLMs in Shiny apps

Three patterns, increasing complexity:

Chat interface — a conversational UI (shinychat)
Tool calling — let the LLM take actions in your app
Data chat — query your dataframe with natural language (querychat)

Shiny

https://shiny.posit.co/py/

pip install shiny

Basic Shiny application

#| '!! shinylive warning !!': |
#|   shinylive does not work in self-contained HTML documents.
#|   Please set `embed-resources: false` in your metadata.
#| standalone: true
#| components: [editor, viewer]
#| layout: horizontal
#| viewerHeight: 500

from shiny.express import input, render, ui

ui.page_opts(title="Basic Shiny App", fillable=True)

ui.input_slider("n", "Number of points", min=10, max=100, value=50)


@render.text
def summary():
    return f"You picked {input.n()} points."

Chatting with your data

What if users could ask questions about your dataset in plain English?

querychat translates natural language → SQL → runs against a DuckDB table:

from querychat import QueryChat
from querychat.data import titanic

qc = QueryChat(titanic(), "titanic")
app = qc.app()

Docs: https://github.com/posit-dev/querychat

Demo: querychat

code/app-querychat.py

from querychat.express import QueryChat
from querychat.data import titanic
from shiny.express import render, ui

qc = QueryChat(
    titanic(),
    "titanic",
    greeting="Ask me about the titanic dataset!",
)

qc.sidebar()

with ui.card():
    with ui.card_header():
        @render.text
        def title():
            return qc.title() or "Titanic Dataset"

    @render.data_frame
    def data_table():
        return qc.df()
    
# 4. Set some page options (optional)
ui.page_opts(
    fillable=True,
    title="Titanic Dataset Explorer"
)

querychat: AI + data science

Safe: only SQL SELECT statements
Reliable: database engine does the execution
Verifiable: see the generated SQL code
Privacy: only column metadata is shared
- numeric ranges
- categorical levels
- column types
- no raw data is shared

Sidebot pattern

Use an LLM to drive filters in a dashboard via natural language:

User types “show me only lunch orders over $20”
LLM translates to filter parameters
Dashboard updates reactively

Template: shiny create --mode core --github jcheng5/py-databot

Demo: LLMs + Chat

Generative AI Shiny templates: https://shiny.posit.co/py/templates/#generative-ai

SQL is a great communication layer

Upcoming tool: ggsql https://ggsql.org/

Try it: https://ggsql.org/wasm/

See also: querychat + vis docs: https://posit-dev.github.io/querychat/py/visualize.html

VISUALISE FROM ggsql:penguins
DRAW point
    MAPPING
      bill_dep AS x,
      body_mass AS y,
      species AS color

Evals and inspect

How do you check AI output?

from chatlas import ChatOpenAI

chat = ChatOpenAI(system_prompt="You are a math tutor.")

# Manually check each response
chat.chat("What is 15 * 23?")  # Did it get this right?
chat.chat("What is the meaning of life?")  # Did it give a good answer?

15 × 23 = 345

There isn’t one universally agreed meaning of life.

Common answers include: - Happiness - Love and relationships - Growth and learning - Helping others - Creating meaning for yourself

A simple way to put it: the meaning of life is the meaning you choose to give it.

<chatlas._chat.ChatResponse at 0x11c0e1a90>

chatlas + inspect-ai

https://posit-dev.github.io/chatlas/

https://inspect.aisi.org.uk//

pip install inspect-ai

Inspect AI

Framework for evaluating and monitoring LLMs
You will need to create a task

Create a task

dataset: test case input and target responses
solver: chat instance using the Inspect AI evaluation framework
scorer: grade responses

Task: dataset

input,target
"What is 15 * 23?",345
"What is the capital of France?",Paris
"What is 100 / 4?",25

Task: solver + scorer

from chatlas import ChatOpenAI
from inspect_ai import Task, task
from inspect_ai.dataset import csv_dataset
from inspect_ai.scorer import model_graded_qa

chat = ChatOpenAI()

@task
def my_eval():
    return Task(
        dataset=csv_dataset("code/my_eval_dataset.csv"),
        solver=chat.to_solver(),
        scorer=model_graded_qa(model="openai/gpt-4o-mini")
    )

Get eval results

inspect eval code/my_eval.py

inspect view

From Chat to Coding Agents

Recap: tool calling

The LLM decides when to call a function and what arguments to pass
Your code — not the LLM — actually executes the function
The result is appended back into the conversation as another message

What’s a harness?

The outer program driving the loop: send messages → get a tool call → run it → feed the result back → repeat
Manages conversation state, permissions, and the available tool surface
Claude Code, Codex, and similar coding agents are all harnesses built around this loop

Skills: markdown prompts, not code

A skill is a folder with a markdown file: a short name + description, plus a body of instructions
No new capability is added — it’s a structured prompt the model can choose to read
Think of it like a recipe card: the harness doesn’t run it, the model does

How does the harness decide which skill to load?

Every installed skill’s name + description sits in context all the time — a short, standing table of contents
The model reads the task and the descriptions and decides which one (if any) applies
Only then is the skill invoked — and its full body loaded into context

Is that RAG?

RAG: embed the query, search a vector index, retrieve the top-k semantically similar chunks from a large external corpus
Skills: the entire list of short descriptions is already sitting in context — no embedding step, no similarity search, no external index to query
Loading the full skill is a deliberate, explicit action (a tool call), not a retrieval step

Closer to lazy imports than search: the signatures are always visible, the body loads on demand.

When the list gets too big

The “just enumerate everything” approach only works while the number of installed skills/tools is small enough to fully list in context
At larger scale, harnesses add a search step — a tool that queries over many candidate tool/skill definitions and loads only the matches into context
That search step is retrieval — this is where the RAG comparison actually starts to fit

Putting it together

Everything you saw today, scaled up:

Conversation loop (chatlas) → agentic loop (harness): decide, act, observe, repeat
One tool (a Python function) → many tools: read/edit files, run shell commands, search the web
One system prompt → a library of skills: reusable, on-demand instructions for specific workflows
A chat window → Claude Code / Codex: an LLM + a harness + tools + skills, working inside your actual codebase

Same idea, bigger loop

{"role": "system",    "content": "You are a coding agent."},
{"role": "user",      "content": "Fix the failing test."},
{"role": "assistant", "content": "", "tool_calls": [{"name": "bash", "arguments": {"cmd": "pytest -x"}}]},
{"role": "tool",      "content": "FAILED test_foo.py::test_bar - AssertionError"},
{"role": "assistant", "content": "", "tool_calls": [{"name": "edit_file", "arguments": {"...": "..."}}]},
{"role": "tool",      "content": "file updated"},
{"role": "assistant", "content": "Fixed — the off-by-one in `foo()`."}

Same request/response/history pattern from the very first slide — just with more tools and a longer memory.

LLMs + Data Science

Summary

LLMs can be powerful tools for data science
We are on the “jagged frontier” of LLM capabilities
Use tools and frameworks to improve reliability and trustworthiness
Combine LLMs with dashboards for more flexible interactive data exploration

Thank you!

https://github.com/chendaniely/2026-07-06-pyladies-llm

@chendaniely