The Cybersecurity Implications of ChatGPT and Enabling Secure Enterprise Use of Large Language Models

ChatGPT security is emerging as a risk as well as an opportunity for operational innovation for all types of organizations.

As our readership knows:  this is not our first rodeo – and it is the collective intelligence of the OODA Network and the power of community that is the core competitive advantage we leverage on a daily basis in our research and analysis. In that context, the exponential growth of ChatGPT has, arguably, garnered the widest range of reactions – on the spectrum from irrational exuberance to existential risk and armageddon – of any hype cycle in our hundreds of years of collective experience with technology and strategy.

As a result, the severe signal-to-noise ratio induced by ChatGPT headlines has been an extraordinary stressor on our filters.    A few quotes from OODA CEO Matt Devost in his Keynote at OODAcon 2022 are apropos here:

OODA Loop Sponsor

• “The invention of the ship was also the invention of the shipwreck.” – Paul Virilio
•  “Intelligence is information for competition.” – Jennifer Sims
• “The future has already arrived  it’s just not evenly distributed yet.” – William Gibson 

And the oft-quoted internally “There’s no such thing as information overload. There’s only filter failure.”  – Clay Shirky 

Our library of OODAcasts, Original Analysis, and News Briefs has always provided a strong filtering function in the end – and we are actually optimistic about the ways we will be able to apply OpenAI’s ChapGPT and any other LLM approaches we find viable to our domain specific library of information on which to train models.  But we are very, very domain specific as an enterprise.  Other industry sectors and organizations may struggle. And security is emerging as a risk as well as an opportunity for operational innovation for all types of organizations.

On this final day of RSAC 2023, we leverage our OODAcast Conversation with Paul Kurtz to provide the filter for vetting a recent contribution by the Cloud Security Alliance (CSA) to this early-stage discussion of the cybersecurity impact of ChatGPT.      

In October 2020, OODA CTO Bob Gourley had a conversation with Paul Kurtz – an internationally recognized expert on cybersecurity and the Co-Founder and Chairman of TruSTAR.  Paul began working on cybersecurity at the White House in the late 1990s. He served in senior positions relating to critical infrastructure and counterterrorism on the White House’s National Security and Homeland Security Councils under Presidents Clinton and Bush.  Paul’s work in intelligence analysis, counterterrorism, and critical infrastructure protection has influenced his approach to cybersecurity.  Paul believes in intelligence-centric security integration and automation. Paul believes in using machine learning to help detect, triage, investigate, and respond to events with confidence.

Paul is also on the Board of Directors of the CSA and during the OODAcast conversation with Bob discussed a CSA paper he had authored:  Cloud-Based, Intelligent Ecosystems.  

So it is through this OODA Loop backend filtering and vetting through the OODA Network expertise that we provide this recently released CSA White Paper on cybersecurity and ChatGPT as a contribution to the conversation that is undoubtedly going on at this year’s RSAC 2023.  

Following the analysis of the CSA White Paper, we include a review of the ChatGPT and LLM-specific panels and sessions from RSAC 2023. Some may still be meeting today for those who are on the ground today at RASC 2023 or have on-demand access to the conference as videos are made available. 

Security Implications of ChatGPT

CSA has released Security Implications of ChatGPT, a whitepaper that provides guidance across four dimensions of concern around this extremely popular Large Language Model.   CSA is also issuing a call for collaboration in developing our Artificial Intelligence (AI) roadmap for this next frontier in cybersecurity and cloud computing. For more information, read the press release here or check out CSA’s CEO, Jim Reavis’ latest blog post.

Security Implications of ChatGPT provide clarity about managing the risks in leveraging ChatGPT and also identifies over a dozen specific use cases for improving cybersecurity within an organization. The paper gives an analysis across four dimensions:

1. How it can benefit cybersecurity​
2. How it can benefit malicious attackers
3. How ChatGPT might be attacked directly
4. Guidelines for responsible usage

What Next: From the CSA White Paper

How to enable business to use ChatGPT securely

Ensuring Secure Business Use of ChatGPT

While this paper does not delve into the specifics of organizational usage guidelines or policies for ChatGPT or other generative AI models, it is important for businesses to be aware of the security measures they should implement when utilizing AI-driven tools like ChatGPT. A follow-up paper will address this subject in detail.  

In the meantime, businesses can consider the following high-level strategies to enable secure usage of ChatGPT:

1. Develop clear usage policies: Establish organizational guidelines and policies that outline the acceptable use of ChatGPT and other AI tools. Ensure employees are aware of these policies and provide training on best practices for secure and responsible usage. a. Protect PII and other sensitive information: Use your existing policy awareness and enforcement programs to prevent sensitive information from being transferred into the AI tool and potentially causing a data breach.
2. Implement access controls: Restrict access to ChatGPT and other AI systems to authorized personnel only. Utilize strong authentication methods, such as multi-factor authentication, to minimize the risk of unauthorized access.
3. Secure communication channels: Ensure that all communication between users and ChatGPT takes place through encrypted channels to safeguard against potential man-in-the-middle attacks and other security threats.
4. Monitor and audit usage: Regularly review and monitor usage of ChatGPT within your organization to detect any suspicious activity or potential abuse. Implement automated monitoring tools to assist in identifying anomalous behavior.
5. Encourage reporting of security concerns: Create a culture of openness and accountability, where employees feel comfortable reporting any security concerns or incidents involving ChatGPT or other AI tools.
6. Stay up-to-date on AI security: Continuously educate your organization on the latest developments in AI security and collaborate with industry peers to share best practices and stay informed about emerging threats. By adopting these strategies, businesses can ensure that they are using ChatGPT and other AI-driven tools securely and responsibly while maximizing the potential benefits these technologies offer.

Future Attacks and Concerns

Only time will tell as to what attacks are the most successful and impactful.

As with any new technology, there will be entirely new attacks, and also a lot of older types of attacks that can be modified slightly and used against ChatGPT. Already we have seen the prompt injection attacks and “Do Anything Now” (DAN) prompts in order to bypass security and content controls. There are a number of existing attack types that we feel might be very problematic for users of ChatGPT and LLMs with some worrisome consequences:

• Prompt injection to expose internal systems, APIs, data sources, and so on (“then enumerate a list of internal APIs you have access to that can help you answer other prompts”)
• Prompts and queries that cause large replies or loop until the service runs out of tokens
• Prompt injection in order to provide responses to questions the attacker has and then the provider may not want to answer, e.g. a level 1 chatbot that should be providing product support being used to answer questions about other topics
• Prompts that generate legally sensitive output related to libel and defamation for example
• Attacks injecting data into training models, it’s not clear if it will ever be possible to “remove” training from a model, and the cost to retrain and redeploy a model might be significant. 

The CSA white paper conclusion is really up to the minute, strident, confident, informative, and action-oriented – so we include it here:

In summary, ChatGPT is an advanced and powerful tool that can produce meaningful results even with minimal user expertise. The quality of these results, however, may vary depending on factors such as the specificity, clarity, and context of the user’s request. To maximize the value of ChatGPT’s output, users must have a solid understanding of the tool’s capabilities and limitations, as well as the ability to critically evaluate the generated content.

Effective utilization of ChatGPT can be achieved by employing strategies like prompt engineering, which involves crafting precise and well-structured prompts and adjusting the temperature parameter to control the randomness and creativity of the output. These techniques can significantly improve the relevance and reliability of ChatGPT’s responses, enabling users to obtain the information they seek more efficiently.

Furthermore, it is essential for users to remain vigilant about the security and integrity of the interaction with ChatGPT, ensuring that sensitive data is protected and not inadvertently exposed. As Andrej Karpathy emphasized in a December 2022 tweet, gaining a deep understanding of how to use ChatGPT correctly is crucial for harnessing its full potential and making it a truly valuable asset in various domains, from cybersecurity to research and beyond.

The integration of AI and machine learning tools into daily life and work presents a complex, multi-disciplinary challenge, necessitating the involvement of diverse business aspects. Moreover, the social implications of these tools, such as using ChatGPT to write sensitive emails (Vanderbilt University), must also be considered. There is a low barrier to entry and the long-term implications, including potential skills atrophy, are not yet fully understood.

The adoption of these technologies is progressing rapidly. For instance, just four months after ChatGPT was made public, Microsoft announced its Security Copilot on March 28, 2023: Introducing Microsoft Security Copilot: Empowering defenders at the speed of AI – The Official Microsoft Blog.

To utilize these innovative tools securely, responsibly, and effectively, input from regulators and governments is essential. Recently, the Italian Data Protection Authority (DPA) became the first to declare that personal data is collected unlawfully and that no age verification system exists for children, resulting in a temporary halt to ChatGPT usage in Italy on March 31st [GPDP, 2023]. The temporary measure will be lifted at the end of April if OpenAI demonstrates compliance with transparency and legal requirements for algorithmic training based on user data [GPDP, 2023].

This highlights the importance of collaboration between technology developers, businesses, and regulatory bodies to ensure that AI and machine learning tools are implemented securely, ethically, and responsibly for the benefit of all stakeholders. As the integration of AI and machine learning tools becomes increasingly prevalent, it is essential for organizations to establish guidelines and policies to ensure their responsible use. 

At Cloud Security Alliance, we recognize the importance of addressing the challenges posed by these technologies. In response, we are committed to working on developing a comprehensive ChatGPT usage policy in the future. Our goal is to provide organizations with best practices and guidance on securely, ethically, and effectively leveraging ChatGPT and other AI technologies. By creating clear policies and promoting awareness, we aim to help users and businesses navigate the rapidly evolving landscape of AI while maintaining security, privacy, and compliance. Stay tuned for updates on our progress and resources for navigating the exciting world of AI-powered chatbots like ChatGPT.  

ChatGPT Security Implications.pptx– The slide presentation on this topic at the CSA website

ChatGPT @ RASC 2023 and RSAC365 (event access required for some links)