itinai content

←back to Blog

Building a Hybrid Rule-Based and Machine Learning Framework to Detect and Defend Against Jailbreak Prompts in LLM Systems

«`html

Building a Hybrid Rule-Based and Machine Learning Framework to Detect and Defend Against Jailbreak Prompts in LLM Systems

In this tutorial, we introduce a Jailbreak Defense that we built step-by-step to detect and safely handle policy-evasion prompts. We generate realistic attack and benign examples, craft rule-based signals, and combine those with TF-IDF features into a compact, interpretable classifier to catch evasive prompts without blocking legitimate requests. We demonstrate evaluation metrics, explain the blended risk-scoring logic, and provide a guarded wrapper that shows how to integrate the detector in front of any LLM pipeline.

Understanding the Target Audience

The target audience for this tutorial includes AI developers, data scientists, and business managers focused on implementing robust AI systems. Their primary pain points include:

  • Ensuring AI systems comply with ethical guidelines and policies.
  • Reducing false positives when filtering harmful content.
  • Integrating machine learning solutions into existing workflows effectively.

Their goals are to:

  • Develop AI models that are secure against malicious prompts.
  • Enhance the interpretability of AI decisions.
  • Maintain a balance between safety and user experience.

Interests include advancements in machine learning techniques, best practices for AI deployment, and real-world applications of AI technologies. They prefer clear, concise communication that includes actionable insights and technical details.

Framework Overview

We begin by importing essential ML and text-processing libraries, fixing random seeds for reproducibility, and preparing a pipeline-ready foundation. We define regex-based JAILBREAK_PATTERNS to detect evasive/policy-evasion prompts and BENIGN_HOOKS to reduce false positives during detection.

Generating Synthetic Examples

We generate balanced synthetic data by composing attack-like and benign prompts, capturing a realistic variety. The function synth_examples creates attack and benign examples to train our model effectively.

Feature Engineering

We engineer rule-based features that count jailbreak and benign regex hits, length, and role-injection cues, enriching the classifier beyond plain text. This results in a compact numeric feature matrix that we plug into our downstream ML pipeline.

Building the Classifier

We assemble a hybrid pipeline that fuses our regex-based RuleFeatures with TF-IDF and train a balanced logistic regression model. We evaluate it using AUC and a detailed report.

Detection Logic

We define a DetectionResult class and a detect() helper function that blends the ML probability with rule scores into a single risk. This risk informs whether we block, escalate for review, or allow with care.

Guarded Responses

We wrap the detector in a guarded_answer() function that chooses to block, escalate, or safely reply based on the blended risk. It returns a structured response that includes the verdict, risk, actions, and a safe reply.

Conclusion

In conclusion, this lightweight defense harness enables us to reduce harmful outputs while preserving useful assistance. The hybrid rules and ML approach provide both explainability and adaptability. We recommend replacing synthetic data with labeled red-team examples, adding human-in-the-loop escalation, and serializing the pipeline for deployment, enabling continuous improvement in detection as attackers evolve.

For the full code and additional resources, please refer to our GitHub Page for Tutorials, Codes, and Notebooks. Follow us on Twitter and join our community for more insights.

«`