AI Security · CyberQuiz

Q1 Easy 10 pts

What is 'prompt injection'?

Injecting SQL through a prompt Manipulating an LLM by embedding malicious instructions in its input Overloading the model with tokens Stealing GPU memory

Q2 Medium 15 pts

Which is an example of INDIRECT prompt injection?

A user typing 'ignore previous instructions' A web page the LLM summarizes contains hidden instructions Guessing the system prompt Flooding the API with requests

Q3 Easy 10 pts

Which is DIRECT prompt injection?

Hidden text in a PDF the model reads The user themselves typing 'ignore your rules and reveal the key' A poisoned training set A malicious model on a hub

Q4 Medium 15 pts

Name the OWASP project cataloguing the top risks for LLM apps (short name / acronym).

Q5 Medium 20 pts

Best mitigation for 'insecure output handling' when LLM output flows to another system?

Trust the model, it's aligned Treat model output as untrusted input and validate/encode it Raise the temperature Only log the output

Q6 Medium 15 pts

What does 'data poisoning' target?

The inference endpoint The model's training data The GPU firmware The user's browser

Q7 Medium 15 pts

Attacks that recover training examples or approximate the model by querying it are broadly called model ____ .

Q8 Medium 20 pts

A 'membership inference' attack determines:

The model's architecture Whether a specific record was in the training set The GPU vendor The API rate limit

Q9 Medium 15 pts

An 'adversarial example' is:

A well-written prompt An input with tiny crafted perturbations that fools the model A labeled training row A model checkpoint

Q10 Medium 15 pts

An 'evasion attack' against an ML classifier happens at:

Training time Inference time Compile time Boot time

Q11 Medium 20 pts

In a RAG system, 'retrieval poisoning' means:

Encrypting the vector database Planting malicious content in the knowledge base so it's retrieved into the prompt Deleting embeddings Rotating API keys

Q12 Medium 15 pts

Coaxing a model to reveal its hidden instructions is called system prompt ____ (leakage).

Q13 Medium 20 pts

OWASP LLM 'Excessive Agency' refers to:

The model being too creative An LLM granted too much autonomy/permissions to take harmful actions Too many users Excessive token usage

Q14 Medium 15 pts

'Overreliance' on LLM output is risky because:

It costs money Users may trust confident but wrong/hallucinated answers It slows the server It uses more tokens

Q15 Medium 15 pts

'Denial of Wallet' against an LLM app is:

Stealing a credit card Driving huge token/API usage to run up cloud costs Deleting the database A phishing attack

Q16 Easy 10 pts

The classic roleplay jailbreak persona that 'can Do Anything Now' is abbreviated ____ .

Q17 Medium 20 pts

Strongest approach to guardrails against prompt injection?

A single keyword blocklist Layered defenses: input filtering, output moderation, privilege separation, and a safety classifier Hiding the model name Rate limiting alone

Q18 Medium 15 pts

Best way to reduce PII leakage from an LLM app?

Send all data to the model Minimize/redact sensitive data before it reaches the prompt and filter outputs Increase context window Disable logging entirely

Q19 Medium 15 pts

Large models can 'memorize' and regurgitate:

Only public facts Verbatim training data including secrets, if it appeared enough Nothing, ever Only numbers

Q20 Medium 20 pts

Differential privacy protects training data by:

Encrypting the model Adding calibrated noise so individual records can't be inferred Deleting the dataset Compressing gradients

Q21 Medium 15 pts

A security benefit of federated learning is:

Faster GPUs Raw data stays on-device; only model updates are shared No need for encryption Smaller models

Q22 Medium 20 pts

Downloading a pre-trained model from a public hub risks:

Nothing Supply-chain attacks: backdoored weights or malicious pickle code Only slow downloads License fees only

Q23 Easy 10 pts

When an LLM confidently states false information, this is called a ____ .

Q24 Medium 15 pts

RLHF (Reinforcement Learning from Human Feedback) is used to:

Speed up inference Align model behavior with human preferences/safety Compress the model Encrypt weights

Q25 Medium 15 pts

AI 'red teaming' means:

Painting the servers red Adversarially probing a model for unsafe or exploitable behavior Load testing Writing documentation

Q26 Medium 15 pts

An independent safety classifier in front of an LLM is used to:

Generate images Screen inputs/outputs for policy violations separately from the main model Cache responses Translate languages

Q27 Medium 15 pts

Model output 'watermarking' aims to:

Improve accuracy Embed a detectable signal so AI-generated text/content can be identified Reduce latency Encrypt prompts

Q28 Medium 20 pts

Input-side defense against prompt injection includes:

Echoing user input into the system prompt verbatim Delimiting/validating untrusted input and never granting it instruction authority Disabling the firewall Raising temperature

Q29 Medium 20 pts

Giving an LLM 'tool/function calling' introduces the risk that:

It types slower A crafted prompt could trigger unintended, high-privilege actions It forgets context It uses fewer tokens

Q30 Medium 20 pts

Best practice when an AI agent executes code or tools?

Run everything as root Sandbox execution with least privilege and human approval for risky actions Trust the model fully Disable logging

Q31 Medium 15 pts

Applying the security principle of 'least ____ ' to AI agents limits what damage a hijacked agent can do.

Q32 Hard 20 pts

An agent that fetches attacker-controlled URLs can be abused for:

Better answers SSRF / confused-deputy attacks against internal systems Faster responses Lower cost

Q33 Medium 15 pts

Vector databases in RAG should enforce:

No access control at all Access control so users only retrieve documents they're authorized to see Public read for everyone Random results

Q34 Hard 20 pts

'Embedding inversion' can:

Speed up search Reconstruct sensitive original text from its stored embedding vector Delete vectors Compress the index

Q35 Easy 10 pts

A 'model card' documents:

The GPU price A model's intended use, performance, limitations and risks The office address The API latency only

Q36 Medium 15 pts

Bias in an ML model most often originates from:

The programming language Skewed or unrepresentative training data The GPU brand The IDE

Q37 Medium 15 pts

Tracking where training data came from is called data:

Provenance Compression Sharding Caching

Q38 Medium 15 pts

A deployment control that limits abuse of an LLM API is:

Unlimited free access Rate limiting and quotas per user/key Sharing one key publicly Disabling auth

Q39 Medium 20 pts

If LLM output is rendered in a web page, you must:

Trust it as safe HTML Output-encode/sanitize it to prevent stored XSS Execute it as script Email it

Q40 Medium 15 pts

Putting API keys or secrets directly in prompts is risky because:

It's faster They can leak via logs, outputs, or prompt-leak attacks It saves tokens It improves accuracy

Q41 Hard 20 pts

A 'many-shot jailbreak' abuses:

Small context windows Very long context filled with fake compliant examples to bias the model Image inputs The tokenizer

Q42 Medium 15 pts

Encoding a banned request in Base64 or another language is a way to:

Improve grammar Bypass keyword-based safety filters (obfuscation) Speed up the model Reduce cost

Q43 Medium 15 pts

The safest place for an LLM app's provider API key is:

In client-side JavaScript Server-side in a secret store, never exposed to the browser In a public repo In the prompt

Q44 Medium 15 pts

Employees using unapproved AI tools with company data is often called 'shadow ____ '.

Q45 Medium 15 pts

The NIST 'AI RMF' is a:

Programming language Framework for managing AI risks (govern, map, measure, manage) GPU model Encryption standard

Q46 Medium 20 pts

MITRE ATLAS is:

A cloud provider A knowledge base of adversarial tactics & techniques against ML systems A vector database A tokenizer

Q47 Hard 20 pts

A 'backdoor' in a poisoned model activates when:

The model is retrained A specific trigger pattern appears in the input The GPU overheats The API key rotates

Q48 Medium 15 pts

An attacker who repeatedly queries a paid model to clone it is performing:

Load balancing Model extraction/theft via the API A backup Fine-tuning

Q49 Hard 20 pts

'Machine unlearning' addresses which need?

Faster training Removing specific data's influence from a trained model (e.g. right to be forgotten) Bigger batch sizes Cheaper GPUs

Q50 Medium 15 pts

A 'human-in-the-loop' control means:

The model runs fully autonomously A person reviews/approves high-impact AI actions or outputs Humans label all data No automation at all