In this post (the 2nd in this series on Tensors and Language Models), we will be interested in collections of tables or dataframes that look something like:

We will call these databases. This example database comes from the paper on quantifying the knowledge capacity of attention layers.

By the end of this post, we will see how to define a tensor associated to a database. But first, we will begin by defining a matrix associated to a function $f: X \to Y$.

This is the first in a series of posts explaining and expanding on the ideas introduced in the following papers:

• Paying attention to facts: quantifying the knowledge capacity of attention layers
  [arXiv:2502.05076]

• ‘Generalization is hallucination’ through the lens of tensor completions
  [arXiv:2502.17305]

I’ve been participating in the Kan Extension Seminar II, and this week it’s my turn to post about Jon Beck’s “Distributive Laws” at the n-Category Cafe!

The post uses lots of string diagrams for monads, resulting in pictures like the following:

See you there!

In this post, I’ll demonstrate 3 ways to define non-commutative rings in Sage. They’re essentially different ways of expressing the non-commutative relations in the ring:

1. Via g_algebra: define the relations directly
2. Via NCPolynomialRing_plural: define a pair of structural matrices
3. Via a quotient of a letterplace ring: define the ideal generated by the relations (only works for homogeneous relations)