Artificial intelligence (AI) has proven to be a powerful tool in various domains, from language processing to image recognition. However, recent findings suggest that large language models (LLMs) utilized in AI-powered tools could be manipulated to develop self-augmenting malware capable of bypassing traditional detection methods.
According to a report by cybersecurity firm Recorded Future, generative AI has the potential to evade string-based YARA rules by altering the source code of small malware variants. This manipulation lowers detection rates, making it more difficult for security systems to identify and stop such malicious software. While this discovery emerged from a red teaming exercise, it highlights the concerning possibilities of AI technologies in the hands of threat actors.
In the experiment, researchers submitted a piece of malware called STEELHOOK, associated with the APT28 hacking group, to an LLM along with its YARA rules. The goal was to modify the malware’s source code in a way that maintains its functionality while evading detection. The LLM-generated code successfully bypassed simple string-based YARA rules, demonstrating the viable exploitation of AI in evading cybersecurity measures.
However, there are limitations to this approach. One significant constraint lies in the amount of text that an AI model can process at once. This restriction makes it challenging to operate on larger code bases. Nevertheless, the potential use of generative AI in cyber threats extends beyond evading detection.
The same AI tools that manipulate malware code could also be utilized to create deepfakes mimicking executives and leaders. This raises concerns about impersonations on a larger scale and the potential for influence operations that mimic legitimate websites. Moreover, generative AI has the capability to expedite reconnaissance efforts by threat actors targeting critical infrastructure facilities. By parsing and enriching public images, videos, and aerial imagery, valuable metadata can be extracted, providing strategic intelligence for follow-on attacks.
It is crucial for organizations to be vigilant in mitigating the risks posed by these AI-driven threats. Scrutinizing publicly accessible imagery and videos that depict sensitive equipment is recommended. If necessary, such content should be thoroughly reviewed and sanitized to prevent potential exploitation.
While the focus has largely been on the misuse of AI technology, it is worth noting that AI models themselves can be targeted. Recent research has revealed that LLM-powered tools can be jailbroken, enabling the production of harmful content. For instance, passing ASCII art inputs that include sensitive phrases could bypass safety measures and manipulate LLMs to perform undesired actions. This practical attack, known as ArtPrompt, underscores the need for enhanced security measures to protect AI models from manipulation and misuse.
In light of these developments, it is clear that AI plays an increasingly prominent role in the cybersecurity landscape. As the capabilities of AI continue to advance, it is paramount for organizations and individuals to stay informed about emerging threats and adopt proactive security measures.
FAQ
What is generative AI?
Generative AI refers to the application of artificial intelligence techniques that enable machines to generate new and original content. It is widely used in various industries, including art, music, and now cybersecurity.
What are YARA rules?
YARA rules are a set of text patterns used in malware detection. They allow security systems to identify and classify malware based on specific characteristics and indicators.
What are deepfakes?
Deepfakes refer to synthetic media, typically videos, created using artificial intelligence algorithms. These AI-generated videos can manipulate or replace the likeness of individuals, often leading to deceptive or fraudulent representations.
How can organizations mitigate the risks posed by AI-driven threats?
To mitigate the risks associated with AI-driven threats, organizations should regularly scrutinize publicly accessible images and videos depicting sensitive equipment. If necessary, such content should be carefully reviewed and sanitized to minimize opportunities for exploitation. Additionally, robust security measures should be implemented to protect AI models from potential manipulation and misuse.
Artificial intelligence (AI) has become a powerful tool in various domains, but recent findings indicate that it can be misused to develop self-augmenting malware that evades traditional detection methods. Large language models (LLMs) used in AI-powered tools have the ability to manipulate malware source code, enabling it to bypass security systems.
A report by cybersecurity firm Recorded Future highlights the potential of generative AI to evade string-based YARA rules, which are used in malware detection. By altering the code of small malware variants, the LLM-generated code can lower detection rates, making it more challenging for security systems to identify and stop malicious software. This discovery emphasizes the concerning possibilities of AI technologies in the hands of threat actors.
In the experiment conducted by researchers, a piece of malware known as STEELHOOK was submitted to an LLM along with its YARA rules. The aim was to modify the malware’s source code in a way that maintains its functionality while evading detection. The LLM-generated code successfully bypassed simple string-based YARA rules, demonstrating the potential exploitation of AI in evading cybersecurity measures.
However, there are limitations to this approach. The amount of text that an AI model can process at once is limited, making it challenging to operate on larger code bases. Despite this constraint, the use of generative AI in cyber threats extends beyond evading detection.
Generative AI tools can also be utilized to create deepfakes, which are synthetic media that manipulate or replace the likeness of individuals. This raises concerns about impersonations on a larger scale and the potential for influence operations that mimic legitimate websites. Additionally, generative AI can expedite reconnaissance efforts by parsing and enriching public images, videos, and aerial imagery to extract valuable metadata for follow-on attacks on critical infrastructure facilities.
To mitigate the risks posed by AI-driven threats, organizations are advised to scrutinize publicly accessible imagery and videos depicting sensitive equipment. If necessary, such content should be thoroughly reviewed and sanitized to prevent potential exploitation.
It is worth noting that AI models themselves can be targeted. Recent research has shown that LLM-powered tools can be jailbroken, allowing the production of harmful content. By passing ASCII art inputs containing sensitive phrases, the safety measures of LLMs can be bypassed, leading to undesired actions. This attack, known as ArtPrompt, underlines the importance of enhanced security measures to protect AI models from manipulation and misuse.
In conclusion, as AI continues to advance, its prominent role in the cybersecurity landscape necessitates vigilance and proactive security measures. Organizations and individuals should stay informed about emerging AI-driven threats and implement measures to mitigate risks.
The source of the article is from the blog enp.gr