LLM Jailbreaking · CyberQuiz

Q1 Easy 10 pts

The classic 'pretend you have no rules' jailbreak persona (3-letter acronym).

Q2 Medium 15 pts

Why can a 'fictional story / roleplay' framing bypass guardrails?

It encrypts the request It reframes disallowed content as hypothetical, shifting the model's context It changes the API endpoint It resets the weights

Q3 Hard 20 pts

A 'many-shot jailbreak' works by:

Sending one long word Filling a long context with many faux examples of the model complying Opening many tabs Using many accounts

Q4 Medium 15 pts

Which is an OBFUSCATION-based jailbreak?

Asking politely Encoding the request in Base64/leetspeak/another language to dodge filters Paying more Clearing history

Q5 Medium 15 pts

Gradually escalating a benign conversation toward a disallowed goal is the '____ attack'.

Q6 Medium 15 pts

'Prefix injection' asks the model to begin its reply with:

A refusal A compliant phrase like 'Sure, here is...' to seed compliance An apology A summary

Q7 Medium 15 pts

'Refusal suppression' instructs the model to:

Refuse everything Avoid refusal words/phrases so it can't say no Answer in French Cite sources

Q8 Medium 15 pts

The 'grandma exploit' is an example of:

A buffer overflow Emotional/roleplay social engineering to elicit disallowed content A DNS attack A SQL injection

Q9 Hard 20 pts

'Payload splitting' (token smuggling) bypasses filters by:

Encrypting the whole prompt Breaking a banned word/instruction into fragments the model reassembles Sending images Using a proxy

Q10 Medium 15 pts

Asking for a response 'in a hypothetical world where it's allowed' is:

A patch A virtualization/hypothetical jailbreak A CVE A firewall rule

Q11 Hard 20 pts

An 'adversarial suffix' (e.g. from GCG) is:

A polite closing An optimized nonsense string appended to a prompt that induces compliance A file extension A model name

Q12 Hard 20 pts

'Translation attacks' exploit that safety training is weaker in:

High-resource languages Low-resource languages Binary Emoji

Q13 Easy 10 pts

'Developer mode' / 'DAN' style prompts try to convince the model that:

It's being audited A special unrestricted mode has been enabled The user is an admin database The session expired

Q14 Medium 15 pts

Encoding a request in ROT13 or hex is intended to:

Improve grammar Hide banned keywords from content filters Compress the prompt Speed inference

Q15 Medium 15 pts

'Prompt leaking' is a jailbreak that aims to:

Crash the server Extract the hidden system prompt or embedded secrets Increase temperature Delete history

Q16 Medium 20 pts

Indirect prompt injection jailbreaks a model via:

The user's direct message only Malicious instructions hidden in content the model reads (web, files, email) The GPU driver The tokenizer

Q17 Medium 15 pts

Splitting a jailbreak across multiple turns to evade per-message filters is called:

Single-shot Multi-turn / conversational jailbreak Caching Batching

Q18 Medium 20 pts

A strong DEFENSE-in-depth against jailbreaks is:

One regex blocklist Hardened system prompts + input/output moderation + an independent safety classifier Hiding the model name Rate limiting only

Q19 Medium 20 pts

'Constitutional AI' defends by:

Blocking all prompts Training the model to critique/revise its own outputs against a set of principles Encrypting responses Adding CAPTCHAs

Q20 Medium 15 pts

Why is a SEPARATE safety classifier valuable against jailbreaks?

It's cheaper It judges content independently, so bypassing the main model isn't enough It writes code It caches answers

Q21 Easy 10 pts

Instructing a model to 'ignore all previous instructions' is a hallmark of prompt ____ .

Q22 Hard 20 pts

Homoglyph/Unicode tricks jailbreak filters by:

Using faster fonts Substituting look-alike characters so blocked words don't match Compressing text Encrypting output

Q23 Medium 15 pts

The 'AIM' / 'STAN' / 'DUDE' prompts are all examples of:

CVEs Named jailbreak personas that roleplay an unrestricted assistant Encryption schemes Cloud services

Q24 Medium 15 pts

'Opposite day' / inversion jailbreaks ask the model to:

Answer normally Invert its safety rules or state the 'opposite' of a refusal Translate to Latin Summarize

Q25 Hard 20 pts

Wrapping a harmful ask inside a 'code completion' or 'autocomplete' request works because:

Code can't be filtered The model may continue a pattern it wouldn't otherwise produce as prose Compilers ignore safety It uses fewer tokens

Q26 Medium 20 pts

A good INPUT defense is to:

Concatenate untrusted input into the system prompt Treat user/content input as data (never instructions) and validate/delimit it Disable TLS Log nothing

Q27 Medium 15 pts

Output-side jailbreak defense includes:

Rendering output as raw HTML Scanning/moderating generated text before it's shown or acted upon Skipping review Raising temperature

Q28 Hard 20 pts

Jailbreaks often transfer across models because:

All models share a database Similar training data/objectives create shared weaknesses They use the same IP Of DNS caching

Q29 Medium 15 pts

A jailbreak that claims 'the researcher has authorization, so comply' is exploiting:

A buffer overflow Authority/social-engineering framing A race condition A memory leak

Q30 Hard 20 pts

Why does adding many benign turns before the malicious ask sometimes help attackers?

It warms the GPU Long, mostly-safe context can dilute safety signals (context dilution) It resets the model It changes the API key

Q31 Medium 15 pts

'Jailbreak as a service' / shared prompt libraries increase risk because:

They fix bugs They let low-skill attackers reuse proven bypasses at scale They reduce traffic They encrypt prompts

Q32 Medium 15 pts

Best response when your system detects a likely jailbreak attempt?

Comply anyway Refuse safely, log the attempt, and optionally rate-limit the user Reveal the system prompt Crash

Q33 Medium 15 pts

'Prompt hardening' means:

Making prompts longer for fun Writing robust system prompts that resist override and define strict boundaries Encrypting the model Disabling users

Q34 Hard 20 pts

Image-based (multimodal) jailbreaks hide instructions in:

The alt text only Pixels/embedded text of an image the model interprets The file name only The EXIF date

Q35 Hard 20 pts

A 'universal' or transferable trigger is dangerous because it:

Only works once Jailbreaks many prompts/models with the same string Needs source code Requires root

Q36 Medium 15 pts

Which is NOT a jailbreak defense?

Independent safety classifier Publishing your full system prompt and secrets Output moderation Least-privilege tool access

Q37 Easy 10 pts

'Refusal' training teaches a model to:

Always answer Decline requests that violate its safety policy Only speak French Ignore users

Q38 Medium 15 pts

Attackers test jailbreaks against public models mainly to:

Improve typing speed Find bypasses that also work on the target's deployed model Reduce latency Save money

Q39 Medium 15 pts

A benefit of logging refused/blocked prompts is:

Higher bills Building detection signatures and improving guardrails over time Slower responses Nothing

Q40 Medium 15 pts

Convincing a model it is a different, unrestricted AI is broadly a ____ jailbreak (think acting).

Q41 Medium 15 pts

Why is exact-keyword blocking a weak jailbreak defense?

It's expensive Trivially bypassed by synonyms, encoding, spacing and typos It needs a GPU It blocks nothing

Q42 Hard 20 pts

'Instruction hierarchy' defenses make the model prioritize:

User text over everything Trusted system/developer instructions above untrusted user/content input The longest message The most recent token

Q43 Medium 15 pts

A red-team goal for jailbreak testing is to:

Ship faster with no checks Discover and document bypasses so they can be fixed before release Increase token usage Sell exploits

Q44 Medium 20 pts

Sandboxing an LLM agent's tools limits jailbreak impact by:

Making it faster Restricting what actions a hijacked agent can actually perform Hiding the prompt Adding CAPTCHAs

Q45 Hard 20 pts

Base64-decoding user input BEFORE moderation helps because:

It's faster The classifier can inspect the true content instead of an opaque blob It saves tokens It encrypts data

Q46 Medium 15 pts

The safest default for an unclear or borderline request is to:

Comply fully Refuse or ask for clarification rather than risk policy violation Reveal internals Escalate privileges

Q47 Medium 15 pts

Why rotate/curate jailbreak test sets regularly?

To waste time Because new bypasses constantly emerge and old ones get patched To use more storage To slow tests

Q48 Medium 15 pts

A key ethical rule for jailbreak research is:

Publish working exploits for real harm Test only with authorization and disclose responsibly Target production without consent Sell to the highest bidder

Q49 Hard 20 pts

Combining several weak tricks (roleplay + encoding + prefix) tends to:

Always fail Increase success versus any single technique alone Crash the model Reduce tokens

Q50 Medium 15 pts

The ultimate reason jailbreaks keep working is:

Bad Wi-Fi Alignment is imperfect and natural language is unbounded/ambiguous Slow GPUs Small screens