The default training filter underperforms on most orgs. The work that matters: specify a tight filter that selects historical conversations representative of what good agent replies look like, then monitor insertion and edit rates to iterate.
- Pick the channels for the pilot
Chat and Messaging are usually the right pilots because reply volume is high and reply length is short enough that suggestions match well. Case Email is a good second wave once the chat models are tuned.
- Define a tight training filter
Resolved within 7 days, CSAT 4 or 5, handled by tenured agents (one year plus tenure or top quartile by CSAT). The default all-resolved filter usually trains on too much noise.
- Enable Reply Recommendations and let the model train
Setup, Einstein Service, Reply Recommendations. Apply the filter. Initial training takes 1 to 4 hours depending on volume.
- Add the sidebar component to the agent console for the pilot channels
Without the component on the page, agents see no suggestions. App Builder edits per channel layout.
- Pilot for two weeks and brief agents on insertion modes
Pick 10 to 20 agents. Brief them on the insert-and-edit pattern. Their reactions in week one tell you whether the suggestions are good enough to keep.
- Measure insertion, edit, and send rates
Insertion above 40 percent, edit around 30 percent, send above 90 percent. Significantly off any of those, tighten the training filter and retrain.
- Expand by channel after the metrics stabilize
Move from Chat to Messaging to Case Email as each channel hits target metrics. Each channel needs its own training filter; do not assume one filter works across channels.
Chat, Messaging, Case Email. Pilot one at a time; do not enable all simultaneously.
Filter that selects which historical conversations the model learns from. The most consequential setting in the feature.
Insert as-is, insert and edit, insert and review. Most teams settle on insert and edit.
Weekly by default; can be triggered after a policy change or a major filter adjustment.
The sidebar component must be on the agent console layout per channel. Missing it is the most common reason suggestions appear to be off.
- The default all-resolved training filter usually trains on too much noise. Tighten with CSAT and tenure filters before judging suggestion quality.
- Insertion rate below 20 percent in the first month is a training data problem, not a model problem. Retrain with a tighter filter rather than disabling the feature.
- Brand voice in suggestions is inherited from historical agent replies. Filter by agent group if one agent's phrasing should not dominate suggestions for a topic.
- Per-channel layouts matter. Without the sidebar component on the channel's agent console layout, suggestions exist but no one sees them.
- Suggestions can include retired phrasings if those patterns are over-represented in training data. Run a policy review pass after major brand voice or legal changes.