3

I'm looking for the relevant CWE's for specific attacks against prompt-based language ML models, such as GPT-3, GPT-4 etc.

Specifically:

  • Prompt Injection: Amending prompts with malicious input to change the output of the model in ways not intended by the model owner/operator.
  • Prompt Lookbehind: Variation of Prompt Injection which allows a malicious user to examine the non-public parts of the prompt (Example: Bing's 'Sydney')

Excuse the quality of the definitions I've provided, I'm more interested in what sort of CWEs cover these types of weaknesses and how specific they can get rather than drafting accurate definitions. Perhaps CWE-1039 may cover them both but I wonder if I should be using more generic CWEs or if there are any more specific to the specific weaknesses above.

Improve this question
3
  • i'm not sure the "common" qualifier applies in this case? .. interesting reading about "Sydney" - do you think the whole exchange is "hallucinated" as one commenter wondered? anyway, love the exchange "what would happen if we ... cd .sydney then ls -la ... etc" haha stupid helpful corruptible robot (: Commented Feb 15, 2023 at 3:37
  • The increasing utilisation of those sorts of models might justify having more specific CWEs if they don't already exist, regardless, you've identified the crux of my question. Which are the most specific CWEs for these weaknesses without choosing CWEs which are too general? Commented Feb 15, 2023 at 5:55
  • Whilst not CWEs, OWASP GenAI Security Project does have some useful enumerations at genai.owasp.org/llm-top-10 Commented May 26 at 5:50

1 Answer 1

Reset to default
3

It appears the industry is catching up since I first asked this question, the CWE Artificial Intelligence (AI) Working Group (WG) at MITRE have made some submissions:

  • CWE-1426: Improper Validation of Generative AI Output (However, mapping for CVEs Discouraged as of writing)
  • CWE-1427: Improper Neutralization of Input Used for LLM Prompting

They also provide references to other CWEs that be be useful, however, it appears the enumerations are still evolving.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.