Logging a meal in Welva nearly always starts with a search box. People type “pâtes”, “skyr”, “pasteque” without the accent, sometimes a brand name. Behind that box sit two sources with nothing in common: ANSES’s Ciqual 2025 table, which describes generic foods with measured values, and Open Food Facts, which describes branded products entered by a community. This is how Welva’s API, a Cloudflare Worker, gets both to answer in a single list, and what I had to fix once it was in production.
Two sources, two licences, one rule
The first constraint shaped everything else. Ciqual is published under France’s Etalab 2.0 open licence, Open Food Facts under the ODbL. I had no wish to end up redistributing a merged database under a muddled licence, so the two sources stay apart everywhere: separate files, separate D1 tables, separate Vectorize namespaces (ciqual and off-hot). They only meet when a response is ranked, in memory, and the merged result is never stored.
The second rule: no model ever produces a calorie figure. A model may understand “a bowl of pasta”; the kcal, protein, carbohydrate and fat values always come from the chosen record, per 100 g. Ciqual is the primary source and Open Food Facts the secondary one, which explains most of the ranking decisions further down.
From ANSES XML to a Vectorize index
Ciqual 2025 is published as XML on Recherche Data Gouv. A Node script downloads the two exports (foods and composition), parses them and writes two deterministic artefacts: a versioned publication (3,484 foods) and a compact array-of-arrays bundle that ships inside the Worker. The timestamp is a mandatory argument, so identical inputs always produce identical files. Values published as “< x”, “traces” or “-” become 0: using half a detection threshold would mean inventing a measurement ANSES never supplied.
The import runs in two passes, D1 first and Vectorize second, in batches of 64. On the D1 side, upserts make a replay safe, and the version is only flagged as current after the last batch and a row count check, so an interrupted run never publishes a partial table. On the Vectorize side, each food becomes a vector keyed by its Ciqual code:
const texts = rows.map(([, nameFr, nameEn]) => `FR: ${nameFr}\nEN: ${nameEn}`);
const { data } = await env.AI.run("@cf/qwen/qwen3-embedding-0.6b", {
documents: texts,
});
await env.FOOD_INDEX.upsert(
rows.map((row, i) => ({
id: String(row[0]), // Ciqual code
namespace: "ciqual",
values: data[i], // 1,024 dimensions
metadata: {
source: "ciqual",
row: JSON.stringify(row),
ciqualVersion: "2025",
},
}))
);The embedding model is Workers AI’s @cf/qwen/qwen3-embedding-0.6b, which outputs 1,024 dimensions, and the index uses cosine similarity. I embed the French and English names in the same document so that search works in both of the app’s languages without doubling the index. The full Ciqual row (names, macros, food group) travels as metadata, so a Vectorize match is enough to render a result without a round trip to D1.
Qwen3-Embedding is asymmetric. Documents are embedded bare; queries carry an instruction: “Given a multilingual food search query, retrieve relevant generic foods and branded food products that match it.” Without it, short queries sit further away from Ciqual labels, which are long and descriptive (“Pâtes sèches standard, cuites, non salées”).
Model and index form a single contract. If I swap the model, even for one that also outputs 1,024 dimensions, both namespaces have to be re-indexed before the new code ships. Otherwise queries and documents no longer share a space, and search quality quietly falls apart without a single error. That warning lives in a comment above the constant, because it is exactly the kind of trap you forget six months later.
The off-hot namespace fills up through use: every Open Food Facts product a user scans or picks is embedded (“name — brand”) and inserted with its record as metadata. The products people actually eat become findable by semantic search, without importing the millions of products in Open Food Facts.
One query, three channels
A search runs three channels in parallel and merges what they return:
- Semantic: embed the query, then query both namespaces (
topKbetween 12 and 24). - Open Food Facts: the Search-a-licious API, with a trimmed field list and a 5.8 s timeout.
- Ciqual lexical: a synchronous in-memory scan of the bundled table, which cannot fail.
const [semantic, off] = await Promise.all([
settle(semanticCandidates(runtime, input, signal)),
settle(withTimeout(() => runtime.searchOff(input.query), 5_800)),
]);
const candidates = [
...(semantic.ok ? semantic.value : []),
...(off.ok ? off.value : []),
...lexicalCiqualCandidates(input.query, input.langs),
];
return {
results: rankFoodCandidates(input.query, candidates).slice(0, input.pageSize),
partial: !off.ok,
};If one Vectorize namespace fails, the other carries on; if Open Food Facts does not answer, the response goes out with partial: true. Only complete responses are cached.
Normalisation is deliberately basic: Unicode decomposition, accents stripped, lower case, anything that is not a letter or digit turned into a space. On top of that sits a very light French stemmer, used only for comparisons: a trailing “s” and then a trailing “e” are dropped, except on words of three letters or fewer (riz, blé and thé stay as they are). That is how “lardons” lands on the “Lardon” record.
I did not write a synonym dictionary. Colloquial phrasing is left to the multilingual embedding, and free-text entry (“describe your meal”) takes a different route: Llama 3.3 70B splits the sentence into foods using a JSON Schema response, and each food then goes through this same search. The whole plan is embedded in a single call, after which the Vectorize queries fan out in parallel.
Ranking without a reranker
There is no reranking model in this search. Ranking starts from the raw score (cosine similarity, the Search-a-licious score, or the lexical channel’s base score) and adds deterministic boosts:
| Signal | Boost |
|---|---|
| The name starts with the query’s words | +0.18 |
| Share of query words found in the name | up to +0.12 |
| A query word matches the brand | +0.30 |
| Raw food (a basic generic record) | +0.015 |
Duplicates are merged on Ciqual code or barcode, keeping the best raw score. Open Food Facts products with no nutrition values at all are dropped. For one- or two-word queries, when Ciqual and Open Food Facts are within 0.05 of each other, Ciqual wins: someone typing “yoghurt” usually wants a generic yoghurt, not whichever branded product happens to come first.
I chose readable rules over a reranker for a practical reason: when a user reports a bad result, I can write a test that reproduces it and fix it, without adding a model call to every keystroke.
What didn’t work
Short words. On day one, “pasteque” returned no watermelon at all. Nothing among the semantic neighbours: a single unaccented word makes too thin a vector next to long labels. Hence the lexical channel, added shortly afterwards: any Ciqual record where every query word is a prefix of a word in its name becomes a candidate, with a modest base score between 0.55 and 0.60 that shrinks with the length of the name. The channel guarantees that the right candidate is present, not where it ranks. The length penalty breaks ties: without it, “Pomme, pulpe, crue” (apple) and “Pomme de terre dauphine, surgelée, crue” (a frozen potato dish) finished level and alphabetical order decided. The channel is capped at 24 entries for short queries that match too many rows.
Embedding variance on near-ties. In a decomposed meal, “pâtes cuites” (cooked pasta) came back as “Pâtes sèches standard, cuites” on one call and “Pâtes fraîches farcies (ex : raviolis, tortellinis), cuites” on the next. When two Ciqual records score that close, similarity stops meaning anything. Below a score gap, I now break the tie on words: the name that starts with exactly the first query word wins, then the one with the fewest missing words (each costs two points) and foreign words. The gap was 0.03 in the first fix; it moved to 0.08 in the rework that introduced stemming, and the tie-break gained a tier: an exact first-word match, before stemming, beats a stemmed one. “Pâtes” (pasta) and “Pâte feuilletée” (puff pastry) differ only by the plural, which the stemmer erases.
Open Food Facts is slow. Search sometimes waited 5.8 s before answering. I added an instant mode: the route serves the semantic and Ciqual lexical results in a few hundred milliseconds, flagged partial: true, under its own cache key (15 minutes, against an hour for complete responses). The app fires both requests in parallel and swaps the list when the complete one lands.
Workers AI cold starts. Measured in production: the first embedding after a quiet spell can take around 5 s, which made the instant response slower than the complete one. It now has a budget:
const semanticTask = options.skipOff
? withTimeout(
(signal) => semanticCandidates(runtime, input, signal),
700, // past this, the Ciqual lexical channel answers alone
options.signal
)
: semanticCandidates(runtime, input, options.signal);After 700 ms, only the lexical channel answers. Without that safety net, the budget would not have been possible.
A cache that froze old rankings. Every complete response is cached for an hour, keyed on the normalised query and language. After a ranking change, users kept seeing the old order. The key now carries a version number that I bump whenever ranking changes; it is on version 3.
What I’d do again / what I’d change
I would keep the strict separation of sources. It cost a bit of merging code, but it makes the licensing question trivial and it forced me to treat Ciqual as the reference. I would also build the lexical channel on day one rather than discover it through a watermelon: semantic search without a lexical fallback fails on precisely the simplest queries.
Three things I would change. First, I lack a set of real, annotated queries to measure ranking against, instead of fixing cases one unit test at a time. Second, the boosts (+0.18, +0.12, +0.30) are hand-tuned; with that measurement set, I would calibrate them. Third, going without synonyms is a bet on the embedding that I have not checked systematically. A small table for common abbreviations would cost almost nothing, and I would add one as soon as the search logs show queries that return nothing useful.
