AI & LLM
17 Apr 2026
Italian NLP — entity weighting — what didn't work
Determining which entities a document fundamentally discusses is a deceptively simple problem. Here's what I tried for Italian — and why none of it worked.
Latest post
When brute force beats HNSW — a DuckDB VSS lesson
TL;DR
- At small N, DuckDB's vectorised columnar scan beats HNSW because round-trip and planner overhead swamp the O(log N) saving; the index is dead weight until ~10k+ items.
- Lesson: a vector index is a property of the query shapes the planner can rewrite, not of the column. Run `EXPLAIN ANALYZE` before trusting the 'index created successfully' message from DuckDB.
Table of contents
- What is HNSW?
- Why I added the index in the first place
- The three queries
- The benchmark
- Why each non-winner loses
- Why brute force wins this size class
- What I changed
- The takeaway