
OpenRouter is “the first LLM marketplace, OpenRouter has grown to become the largest and most popular AI gateway for developers. We eliminate vendor lock-in while offering better prices, higher uptime, and enterprise-grade reliability.” They have all kinds of interesting data about models they are serving (rankings), and piles of big-name and obscure models.
Mapping the Latent Past: Assessing Large Language Models as Digital Tools through Source Criticism
- This article examines how digital historians can use large language models (LLMs) as research tools while critically assessing their limitations through source criticism of their underlying training data. Case studies of LLM performance on historical knowledge benchmarks, oral history transcriptions, and OCR corrections reveal how these technologies encode patterns of whose history has been digitised and made computationally legible. These variations in performance across linguistic and temporal domains reveal the uneven terrain of knowledge encoded within generative AI systems. By mapping this “jagged frontier” of AI capabilities, historians can evaluate LLMs not just as tools but as historical sources shaped by the scale and diversity of their training. The article concludes by examining how historians can develop new forms of source criticism to navigate generative AI’s uneven potential while contributing to broader debates about these technologies’ societal impact.
Tasks
- Finish slides – scroll through notes for links – done
- Check in and ping Sande – done
- 4:30 class – done!
SBIRs
- Change df so that cluster id is a column and see if I can get that to work
- That works nicely. Here’s the code that creates the df:
num_populations = 5
num_samples = 1000*num_populations
l = []
scalar = 5.0
for i in range(num_samples):
c = np.random.randint(0, num_populations)
d = {'cluster': f"c{c}", 'x':np.random.normal()+(float(c)-num_populations/2.0)*scalar, 'y': np.random.normal(), 'z':np.random.normal()}
l.append(d)
df = pd.DataFrame(l)
- Here’s the rendering code:
fig = px.scatter_3d(df,
x='x',
y='y',
z='z',
color='cluster'
)
fig.update_traces(marker=dict(size=3))
And here are the results:





You must be logged in to post a comment.