The decision
A choice over your teams, such as payments, delivery, account, technical and other, and a noul question: "the customer asks to talk to a person". The options can stay in English while customers write in Nepali, and the model gives a probability for every team.
How to build it
- Collect 200 real messages and label each with the team that handled it. Keep all three ways people write: Devanagari, Romanized Nepali ("mero order kaha pugyo?") and Nepali mixed with English.
- Pull a model built with Nepali in mind:
systemone pull sagea/mira. Mira is small enough for a CPU, and its model card shows how to run it. - Run the same messages through Laya's multilingual checkpoint, for example
systemone run noulxp convai-innovations/laya --checkpoint multilingual --request message.json, and through Julia 1. - Measure accuracy for each way of writing separately, then choose the model and the threshold.
- Route above the threshold. Below it, reply in the customer's own script and ask which team they need.
Make it better
Fine-tune on your labelled messages in System One Studio (systemone run studio), starting from Laya's multilingual checkpoint (on a Mac, aac6fef/laya-multilingual-mlx). Keep 50 messages of each way of writing aside, to show the fine-tune beats the base on all three.