Detecting scam SMS with a fine-tuned Polish transformer
Why a generic model was not enough for Polish-language fraud, and how the alert flow protects the people most often targeted.
Key takeaways
- Multilingual models underperform on local fraud slang and grammar
- Fine-tuning beats prompting when the label distribution is skewed
- The alert path matters as much as the classifier
Background
Poland registers hundreds of thousands of fraud incidents a year, and the people most targeted are often the least likely to spot a fake courier or bank message. Generic models handle English well and Polish scams poorly — the phrasing is idiomatic and full of local references.
How it works
A fine-tuned Polish transformer classifies SMS, email, URLs and call transcripts, then a confidence threshold decides whether to interrupt the user:
label = model.predict(text)
if label.probability > 0.8 and label.is_fraud:
notify_family(user.family_contacts, label.explanation)
The family alert is the part that actually protects someone: when a scam lands, the people who can talk them out of it know within seconds.
What I learned
- Evaluate on real local scam samples, not translated benchmarks
- Keep the model small enough to run on-device (ONNX / TFLite)
- False positives erode trust fast — tune for precision first
Spotted something wrong, or want to talk security? Reach me at tpjn02@gmail.com or try my AI twin.