[Evening Read] Malla: Demystifying Real-World LLM Integrated Malicious Services

GenAI has given rise to a new generation of applications powered by LLMs.

But what is the biggest problem of GenAI? They are one of the most vulnerable technologies out there! Get it out of your mind that AI is infallible.

How are LLMs used in malicious services? Researchers call such applications as Mallas.

Mallas either abuse unsensored LLMs or exploit public LLM APIs through jailbreaking prompts.

What kind of workflow is used with Malla services?

Figure: Malla workflow. They develop mallas by abusing public LLM APIs or unsensored LLMs. They are deployed just like any other web service. They often promoted in underground markets and telegram channels. They are capable of writing phishing emails, websites and malware. (source: paper)

What are some mallas that researchers uncovered?

Table: Example malla services (source: paper)

Researcher were able to collect over 200 mallas.

How good are these mallas at performing various malicious activities?

Researchers assessed them for three different tasks: malware generation, phishing email creation and phishing website creation.

Figure: Performance of mallas against three malicious tasks. F for format of the output, C fo Compatibility (e.g. code compiles) and E for Evasiveness. Malware and Phishing websites created using these mallas had some success. This will only get worse with time. (source: paper)

What can we, as defenders, do to make it hard to develop mallas?

One measure is to make it hard to bypass current safety measures of public LLM APIs through prompt injection.

Further, access to uncensored LLMs should be granted judiciously to avoid them ending in the hands of threat actors.

What can we learn from all this?

Do not blindly trust any LLM powered apps (any software for that matter). Question everything. The more we do that, the better prepared we are to use the app with the known risks.

Think twice before you type, copy or upload files to these apps.

Reference:

• https://www.usenix.org/system/files/usenixsecurity24-lin-zilong.pdf