How Generative AI is Transforming Cyber Threats & How to Stay Secure

For an internal event at Cryptomathic, we invited some our clients from the financial and trust service sectors to share their insight on future trends around our solutions. We had expected topics such as post quantum cryptography, the use of confidential computing to secure the transition to cloud, targeted threats from groups that want to destabilise high street banks and the economy to be the number one concern.

To our surprise,  AI came up as the number one concern. The rise of Generative AI (GenAI) tools is a double-edged sword. Whilst these tools have revolutionized content creation and task automation, they have also opened the door to new forms of cyberthreats. Highly realistic synthetic media, such as deepfakes, voice cloning, code injection and forced documents are now being used by malicious actors to undermine security measures.

The implications for fraud detection, particularly in Know Your Customer (KYC) protocols, identity verification processes, secure key generation, reverse engineering or document/transaction manipulation are profound.

The advent of Generative AI has led to a proliferation of cyber threats that can be grouped into several categories:

1. Synthetic Identity Fraud: Deepfakes & Voice Spoofing: Tools like VALL-E and DALL-E can clone vocal patterns and create fabricated media. A survey by Regula reported that 37% of organizations globally have experienced deepfake voice fraud attempts. Synthetic Data Fraud: Deduce noted a 17% rise in synthetic data fraud over the last two years, influenced by generative AI.
2. Financial Fraud: Invoice Fraud: An article from Resecurity published earlier this year is an eye opener on the techniques used but also on evaluated losses incl. $120,000 per incident and over $9 billion in possible losses affecting U.S. financial institutions and their customers since 2016. Other surveys reports businesses affected by invoice fraud with 20% experiencing between 21 and 30 instances of such fraud in the same business.
3. Security Protocol Bypass: Machine learning AI tools: AI models can be trained to mimic user behavior (e.g. CATCHA), trick biometric security systems, identify vulnerabilities in protocols, create malware or inject code that evolves using adaptive algorithms to counteract security measures.
4. Phishing: Large Language Models (LLMs): Generative AI can craft convincing phishing emails that manipulate individuals which can lead to successful social engineering attacks. AI also contribute to phishing automation, thereby ensuring higher returns for criminals.

These examples underscore the growing concern for advanced fraud detection and prevention measures in the face of sophisticated AI-generated forgeries. External sources such as IDology’s 2024 Global Fraud Report provide valuable insights into the prevalence and nature of these threats.

To combat these threats and protect themselves against these sophisticated forms of fraud, applied cryptography is an immense resource for our clients. The vast majority of these attacks can be circumvented by ensuring data/document integrity, authenticity and assurance, confidentiality incl. effective data protection.

Cryptomathic is perfectly positioned to help organizations implement defense measures and mitigation strategies. We empower the development of a secure digital future and our cutting-edge solutions designed to ensure the confidentiality, authenticity, and integrity of digital transactions:

1. In our Digital Identities and Signatures domain: Cryptomathic’s offering in this space allows for the implementation of invoice and document sealing, highly effective against Invoice Fraud but also provided legal certainty and enhanced security with the implementation of effective sole control protocol and What You See Is What You Sign (WYSIWYS) which are essential to defeat LLM based attacks or protocol bypass. Digital signatures and identification platform do not only provide integrity and authenticity but also shifts liability to the trust service provider, ensuring compliance and trust in digital interactions.
2. Comprehensive Key Management: The CrystalKey 360 suite offers a robust set of tools for data protection, code signing, data integrity, crypto agility and encryption. By securing the digital assets at every stage, CrystalKey 360 protects against unauthorized access and tampering, safeguarding the core of business operations. It further helps in ensuring compliance against NIS2, DORA, SHREMS2 and can facilitate the secure transition of sensitive application to the cloud.
3. Mobile App Security: With the prevalence of mobile technology, protecting applications from reverse engineering attacks has become paramount. Cryptomathic’s mobile app security core provides a shield against such threats, ensuring that mobile applications remain secure against sophisticated intrusion attempts and provides assurance protocols, secure storage, runtime application self protection. The solution is used as first defense strategy by leading financial providers and issuers of sensitive wallets.

As GenAI continues to evolve, the potential for AI-enabled financial crimes grows. Firms must remain vigilant and adapt their strategies to protect against these sophisticated forms of fraud. Cryptomathic’s solutions offer a way forward, leveraging the power of cryptography to build a more secure digital world.

If you need a personal consultation or want a deeper dive, please feel free to contact our team of experts.