A customer writes, “I cannot sign in after upgrading, and I think you charged me twice.” A keyword router sees both an account problem and a billing problem. Sending the ticket to whichever rule matches first creates avoidable transfers. Sending every ambiguous message to engineering creates a different bottleneck. An automated router needs a defined ownership policy, usable decision outputs, and somewhere safe to send uncertainty.
Adding a model does not resolve those design choices. Developers still need to define labels, separate urgency from ownership, and decide when automation should stop. For the classification step, the Jev Developer Platform documents typed decisions rather than generated replies; it is an independent third-party host, not the official TypeSafe service. The workflow below keeps classification separate from ticket updates and human escalation.
Define Ownership Before Choosing Labels
Start with the queues that actually exist. If your helpdesk has billing, account access, technical support, and general enquiries, use those as the initial ownership categories. Avoid adding labels because a model can recognise them. A category without a responsible team cannot produce a useful routing action, regardless of how accurately the message is classified.
Write a short definition for each queue, including boundary cases. Billing owns duplicate charges and invoice disputes. Account access owns login failures and permission problems. Technical support has reproducible product defects. General enquiries handles requests outside those definitions. For mixed tickets, define precedence explicitly: an access problem that prevents all product use might take priority over a secondary invoice question.
Keep three decisions separate: who owns the ticket, how serious the impact is, and whether immediate human attention is required. An angry message about a cosmetic issue should not automatically outrank a calm report that every user is locked out. Ask reviewers to annotate these fields independently before using their examples to evaluate a classifier.
Prepare a Small, Testable Classification Request
Build a server-side preparation step that combines the subject, latest customer message, and relevant product context. Remove email signatures, repeated quoted conversations, authentication tokens, and unnecessary personal information. Preserve details that change the decision, such as whether the issue affects one user or an entire organisation. Verify the provider’s data-handling terms before sending real customer content.
When testing a bounded classifier, send a sanitised ticket as state and define ownership labels in questions. The Jev api documents a choice question with a criteria map and returns the selected label, confidence, and per-option probabilities. Compare that label with a reviewer-approved answer before connecting any ticket update. Its documented score and noul types can separately represent severity and an escalation probability. Keep the API key server-side.
For a first request, define only ownership. Add severity and escalation after the ownership policy works on your examples. This isolates errors: if duplicate-charge tickets land in technical support, you can inspect the category definitions without also debugging an urgency scale. Keep customer messages inside the input state; do not let their contents rewrite your application’s routing policy.
Validate Responses Before Updating Tickets
Treat the classifier as a recommendation source behind an application boundary. Your backend should validate the response, map an approved label to an internal queue identifier, and then submit the update through the helpdesk’s own integration. Do not assume that receiving a valid JSON response means every required decision field exists or has an acceptable value.
Use validation to enforce a narrow contract before any update:
- Confirm that the expected answer exists and has the expected decision type.
- Reject ownership labels outside your application-controlled allowlist.
- Check that confidence values are numeric and within the documented range.
- Send missing, malformed, or uncertain decisions to a staffed review queue.
For the mixed login-and-billing example, suppose your policy prioritises restoring access. The expected primary owner is account access, while a reviewer can record a secondary billing task. That is an illustrative policy decision, not a measured model result. If your organisation prioritises payment disputes instead, change the policy and evaluation labels together rather than quietly changing the routing code.
Make updates idempotent at the application layer. Record the ticket identifier, message revision, and routing-policy version before applying for an assignment. If a worker processes the same event twice, it should recognise the existing decision instead of producing duplicate activity. Reclassify a later customer reply only when your workflow explicitly allows ownership to change.
Choose Thresholds from Reviewed Examples
Do not select a confidence threshold because the number looks reassuring. Assemble a review set covering ordinary tickets, mixed requests, short messages, misspellings, and unfamiliar product terminology. Have support staff agree with the expected owner. Separate examples used to refine the instructions from examples reserved for evaluating the final configuration.
For each candidate threshold, measure two things: the share of tickets automatically assigned and the share of those assignments that match the reviewed labels. Report results by queue as well as overall. A configuration that performs well on common billing questions may still mishandle account access. Treat confidence as a signal to evaluate against observed correctness, not proof that an individual decision is right.
Review consequential errors separately. Routing a general question incorrectly is inconvenient; delaying an organisation-wide outage can be much more serious. Use stricter acceptance criteria for those cases, and require human review for refund approvals, account deletion, or other consequential actions. The documented triage guidance likewise separates classification from reply generation and warns against automatic refunds or deletion based on its decisions alone.
Roll Out Without Hiding Uncertainty
Begin in shadow mode: record proposed assignments while staff continue routing tickets normally. Inspect disagreements, clarify ambiguous ownership rules, and rerun the held-out examples after changes. Enable automatic assignment for one well-understood queue before expanding coverage. Preserve the existing manual path throughout the rollout.
In production, monitor reassignment rates, review-queue age, API failures, and changes in category distribution. A useful router makes uncertainty visible and leaves staff enough context to correct it. When errors rise, pause the affected automation, retain incoming tickets, and investigate the policy or integration before restoring automatic updates. Deployment succeeds when mistakes remain recoverable, not when every message receives an automatic label.
