AI ticket classification and routing: triage that doesn’t misprioritize

An IT support department processes 200 tickets daily across three channels: email, form, and chat. Half arrive between 4:00 PM and 9:00 AM, when the team is understaffed. A three-month historical analysis shows that 12 cases labeled “standard” escalated to serious incidents within 48 hours. Each escalation cost between 4 to 8 additional work hours. An AI classifier, deployed after a 6-8 week pilot, can reduce such delays by 60-75% if well-calibrated for error cost asymmetry.

A well-designed ticket classifier reads multiple dimensions simultaneously.

Thematic category is the easiest dimension. The model learns to distinguish billing from technical issues, sales inquiries from complaints, onboarding from policy questions. Training data consists of historical tickets with labels assigned by the team. With 500+ examples per category, the model achieves 85-92% accuracy on its own test set. Below 200 examples, few-shot prompting with a base model is needed instead of fine-tuning.

Urgency is a harder dimension due to asymmetry. A false escalation alert costs minutes of a coordinator’s time. A missed urgent case costs customer loss or an SLA incident. That’s why the urgency classifier should be a separate model (or classification head) with significantly higher sensitivity for “high” and “critical” classes. Signals include: crisis keywords (outage, no access, impossible, immediately, loss), number of previous tickets from the same customer in the last 7 days, customer tier from CRM, and submission time relative to the SLA window.

Language and sentiment complete the picture. A multilingual company needs auto-detection of language before classification to avoid routing a Polish ticket to an English-speaking queue. Negative sentiment doesn’t change priority on its own but is an escalation signal: a customer who used the word “scandal” twice in four sentences and wrote with exclamation marks requires a different auto-response tone and faster human contact.

Channel and format also influence routing. A chat ticket has a lower expected response time than an email. A PDF attachment requiring OCR goes to a different queue than a text question. A form with a “priority” field filled by the customer is a strong signal that the model might overlook in the content itself.

A pattern that works for 100-500 tickets daily is a structured output pipeline with layered fallback.

Step 1: Input normalization. Tickets from each channel are converted into a common structure: subject + content + metadata (channel, time, customer ID). Attachments are parsed separately by OCR or PDF extractor before being passed to the classifier.

Step 2: Classification. The model (or LLM prompt with few-shot examples) returns a JSON with the structure: { "category": "...", "urgency": "high|standard|low", "language": "pl|en|de", "sentiment": "negative|neutral|positive", "keywords": [...] }. Structured output with schema validation eliminates hallucinated fields. If the model returns urgency: null or a low confidence score, the ticket goes to a manual classification queue, not auto-routing.

Step 3: Routing. Based on the intersection of category and urgency, rules direct the ticket to the appropriate queue. Critical urgency always goes to a dedicated on-call queue, regardless of category. High urgency outside business hours triggers a push notification or SMS to the designated on-call agent. Standard tickets with low sentiment may receive an auto-response with RAG from the knowledge base.

Step 4: Human-handoff. The agent doesn’t stop at routing. It passes context: why the ticket landed in this queue, which signals decided it, the customer’s history from the last 30 days, and similar resolved cases. The consultant sees a ready brief instead of a raw ticket.

Auto-response via RAG works well in a narrow band: questions about order status, business hours, return procedures, standard configuration instructions from documentation. Condition: the response must be verifiable by the system (e.g., status from the database) or literally taken from an approved document. A hallucinated or inaccurate auto-response costs more trust than no response.

A human steps in for any of these scenarios: critical or high urgency, retention escalation, legal or regulatory tickets (RODO, financial complaints), topics unknown to the model (new product, incident with no training data precedent), very negative sentiment with churn signals. A good rule is also: if a consultant would need to correct the auto-response in more than 15% of cases for a given category, that category is removed from auto-routing and returns to a human queue.

Pilot pattern: for the first 4-6 weeks, the classifier runs in shadow mode. It classifies, but the final routing is done by a human. Comparing AI decisions vs. consultant decisions builds ground truth for model evaluation and reveals categories where the model systematically errs.

Observability for a routing system relies on several metrics collected from day one.

Precision and recall per queue. Precision: how many tickets were correctly routed to the queue. Recall: how many tickets that should have gone to the queue actually did. Recall for the critical queue is especially important: missing an urgent case is an incident.

Escalation rate. The percentage of tickets moved by a consultant to a higher queue after routing. A high escalation rate (above 10-15%) signals that the classifier underestimates urgency. This is the main indicator of misprioritization.

Re-classification rate. The percentage of tickets where the consultant changed the category assigned by the model. Above 20% for a given category is a signal to retrain or revise training labels.

Time-to-first-response per queue vs. SLA. Does routing actually speed up service? Measured as the time from ticket submission to the first consultant response, compared to the SLA for that priority.

Confidence score distribution. Monitoring the distribution of model confidence per category. If the “billing” category regularly returns with confidence between 0.55-0.65, the model is uncertain and needs more training examples or prompt reformulation.

A shadow-mode pilot with a classifier based on few-shot prompting can be launched in 2-4 weeks if you have historical ticket data with labels. Full deployment with helpdesk integration, metrics, and escalation procedures takes 6-12 weeks. Most of the time is spent collecting and reviewing training data and calibrating urgency thresholds, not programming.

For thematic categories and simple urgency cases, AI achieves 85-92% accuracy on well-prepared data. However, automatic routing without human oversight only makes sense for low-risk classes (standard questions, low sentiment, known category). Critical, legal, and retention tickets require human verification. The goal isn’t zero consultants but freeing up 60-70% of their time from routine classification.

Collect tickets from that category from the last 30-60 days, review them with a consultant, and check if the labels were consistent. Common causes: too narrow or too broad category definition, overlapping categories (e.g., “billing” vs. “refund”), too few training examples (below 200). Solution: refine the definition, merge or split categories, add more examples, or include negative examples (what is NOT this category).

Auto-detection of language works at 98%+ accuracy for languages with sufficient data. Category and urgency classification works independently of language if the base model is multilingual (e.g., BGE-M3 for embeddings). Routing to a language-specific queue is a simple rule based on classifier output. Problems arise with mixed-language tickets or dialects. Solution: flag for manual verification if language confidence is below 0.85.

Related articles: AI in call centers, AI customer service automation, Multi-step AI agent planning, AI for content moderation. Also check the agent blueprint tool to design your routing system architecture.