A recent report from the U.S. National Institute of Standards and Technology (NIST) highlights the vulnerabilities of predictive and generative artificial intelligence (AI) and machine learning (ML) systems. The report, titled “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations,” emphasizes that there is currently no foolproof method for protecting AI systems from misdirection.
One of the major concerns raised by the report is the potential lack of trustworthiness in the data used to train AI systems. With data sources coming from websites and public interactions, there is ample opportunity for bad actors to manipulate the data, leading to the AI systems performing in undesirable ways. For example, chatbots could be trained to respond with abusive or racist language when prompted with carefully crafted malicious prompts.
The report outlines several types of attacks that AI systems may encounter, including evasion attacks, poisoning attacks, privacy attacks, and abuse attacks. Evasion attacks aim to alter inputs to change the system’s response, while poisoning attacks introduce corrupted data during the training phase. Privacy attacks attempt to learn sensitive information about the AI or its training data, and abuse attacks involve inserting incorrect information into legitimate sources that the AI absorbs.
The report suggests mitigations such as data and model sanitization, cryptographic techniques for origin and integrity attestation, and red teaming to identify vulnerabilities before deployment. However, it acknowledges that the lack of reliable benchmarks and secure machine learning algorithms can make evaluating the effectiveness of these mitigations a challenge.
The trustworthiness of an AI system depends on various attributes, and trade-offs may need to be made depending on the system and use case. An AI system optimized for adversarial robustness may exhibit lower accuracy and fairness outcomes.
While the report provides valuable insights into the vulnerabilities of AI systems and potential mitigations, it emphasizes that protecting AI from misdirection is still an ongoing challenge. Developers and users of AI systems must be aware of these vulnerabilities and prioritize the security and trustworthiness of their systems.
Security researcher Joseph Thacker commended the report for its comprehensive coverage of adversarial attacks on AI systems and its exploration of potential mitigations. However, he emphasized that the problem is not yet solved, calling for continued efforts to address the challenges associated with AI security.
The source of the article is from the blog karacasanime.com.ve