“AI for Enterprise”: Lessons Learned from Healthcare, Hugging Face and Clinical Language Models

Healthcare is already in the midst of an AI revolution – with an applied technology market maturity which outpaces most other industry sectors which are in a reactive mode to the AI hype cycle.  Explore these AI healthcare use cases and apply them to your organization using design and systems thinking.  

Based on research in 2016 capturing the M&A activity of the technology sector as they acquired artificial intelligence companies, a trend which emerged (which was worth continuing to track) is that the healthcare and biotechnology industry sectors were the tip of the spear in applied technology use cases in artificial technology.  The ecosystem has matured and innovated since 2016 – with interesting case studies, systems thinking, and platform architectures. In the end, these are enterprise implementations – which can inform artificial intelligence implementation efforts across all industry sectors.

As we continue to sort out what “Enterprise AI” truly looks like, in this post we take a look at large language model use cases in the the clinical healthcare solutions marketplace – which is poised for exponential gorwth and breathtaking innovation in the next 10 years.  

OODA Loop Sponsor

BACKGROUND

What are Clinical Language Models?

(Generated by OpenAI’s ChatGPT) 

Clinical language models are specialized natural language processing (NLP) models designed for use in the field of healthcare and medicine. These models are built upon the foundation of general-purpose language models but are fine-tuned and tailored specifically for tasks and applications within the healthcare domain. They are trained on vast amounts of medical literature, electronic health records (EHRs), clinical notes, and other healthcare-related texts to understand and generate text that is relevant to medical professionals and patients.

Here are some key characteristics and applications of clinical language models:

1. Medical Record Processing: Clinical language models can assist in extracting and organizing information from electronic health records (EHRs), including patient histories, lab results, diagnoses, and treatment plans. This can improve the efficiency of healthcare providers and reduce the risk of errors.

2. Clinical Decision Support: These models can provide clinicians with evidence-based recommendations and information to aid in making clinical decisions. For example, they can suggest treatment options based on the latest medical literature and patient data.

3. Patient Communication: Clinical language models can be used to develop chatbots and virtual assistants that can interact with patients, answer their medical questions, and provide information about conditions, medications, and treatments.

4. Medical Literature Summarization: They can help researchers and healthcare professionals quickly summarize and extract relevant information from the vast amount of medical literature available, making it easier to stay up-to-date with the latest research.

5. Diagnostic Assistance: In some cases, clinical language models can assist in diagnosing medical conditions by analyzing symptoms and medical histories. However, these models are typically used as support tools and not as standalone diagnostic tools.

6. Natural Language Understanding: They are capable of understanding medical terminology, abbreviations, and complex medical concepts, making them valuable for interpreting and generating clinical text.

7. Translation and Multilingual Support: Clinical language models can aid in translating medical information and documents between languages, making healthcare information more accessible globally.

8. Privacy and Security: Given the sensitive nature of healthcare data, clinical language models are often designed with strong security and privacy measures to ensure patient confidentiality and compliance with healthcare regulations like HIPAA in the United States.

It’s important to note that while clinical language models offer many benefits in healthcare, they should be used in conjunction with the expertise of healthcare professionals and should not replace clinical judgment. Additionally, ensuring the accuracy and reliability of these models in clinical settings is an ongoing area of research and development.

Hugging Face:  Explore Biomedical Language Models

As we discussed in the Context On The Nature of Web 3.0,  community building is a hallmark of all previous “build outs” of networked capabilities, with the open source movement figuring prominently.  There are two design elements of community interaction in a platform economy:

Participatory Culture:  Forms of public engagement based more on social and cultural protocols and less on technology as the primary driver.

Collective Intelligence:  A term coined by Pierre Levy which refers to the capacity of virtual communities to leverage the special interests of their community members, normally through collaboration and large-scale discussions.

This is proving true in the nascent stages of the AI for Enterprise buildout.  Hugging Face is emerging as an open source meeting place of the AI Developer community and crucial repository of large language model (LLMs), neural language models (NLMs) and natural language processing (NLP) code. 

Already, biomedical language models have a major presence on the site.  For example, one can fo to Hugging Face for Health, where you can Explore Biomedical Language Models boradly categorized and sorted into subgroups:

• Clinical Language Models
• AI in Biology Demos
• Biology Models
• Biomedical Language Models
• Biomedical Vision-Language Models
• Biochemistry Models
• Privacy-Preserving Biomedical ML
• Synthetic Medical Data
• Biomedical Datasets

In the Clinical Language Models section, there are use cases which include an academic paper and code available at github, including:

StanfordAIMI/stanford-deidentifier-base:  Stanford de-identifier was trained on a variety of radiology and biomedical documents with the goal of automatising the de-identification process while reaching satisfactory accuracy for use in production. Manuscript in-proceedings.  These model weights are the recommended ones among all available deidentifier weights.

Clinical Camel: An Open-Source Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding:  Large Language Models (LLMs) present immense potential in the medical field, yet concerns over data privacy, regulatory compliance, and model stability restrict their widespread adoption. Although the distillation of high-performing closed-source LLMs has proven effective for general tasks, their application in healthcare is limited due to reduced domain knowledge and remnants of alignment behavior hindering clinical tasks.

To address these challenges, we propose Dialogue-Based Knowledge Encoding (DBKE). DBKE enhances models’ implicit knowledge base and primes them for conversational recall, augmenting their conversational capabilities and enabling a soft alignment for subsequent use cases. By transforming dense academic source text into synthetic dialogue, DBKE broadens the model’s knowledge base and enables a soft alignment that guides downstream behaviours.

We present Clinical Camel, an open-source, healthcare-focused conversational model, to showcase the effectiveness of DBKE. Clinical Camel outperforms GPT-3.5 on the United States Medical Licensing Examination (USMLE) Step 1 and Step 3 with scores of 53.2 % and 58.2 %, respectively, compared to GPT-3.5’s scores of 36.1 % and 55.7 %. Clinical Camel adeptly handles multi-stage clinical case problems, provides adaptive counseling, and generates clinical notes. However, it is prone to hallucinations, which pose a significant obstacle in safety-critical settings. The performance of Clinical Camel underscores the importance of continued research and development of open-source models for the safe and effective integration of LLMs in healthcare settings.

ChatDoctor: A Medical Chat Model Fine-tuned on LLaMA Model using Medical Domain Knowledge:  Recent large language models (LLMs) in the general domain, such as ChatGPT, have shown remarkable success in following instructions and producing human-like responses. However, such language models have not been learned individually and carefully for the medical domain, resulting in poor diagnostic accuracy and inability to give correct recommendations for medical diagnosis, medications, etc. To address this issue, we collected more than 700 diseases and their corresponding symptoms, recommended medications, and required medical tests, and then generated 5K doctor-patient conversations.

By fine-tuning models of doctor-patient conversations, these models emerge with great potential to understand patients’ needs, provide informed advice, and offer valuable assistance in a variety of medical-related fields. The integration of these advanced language models into healthcare can revolutionize the way healthcare professionals and patients communicate, ultimately improving the overall quality of care and patient outcomes. In addition, we will open all source code, datasets and model weights to advance the further development of dialogue models in the medical field. In addition, the training data, code, and weights of this project are available at: https://github.com/Kent0n-Li/ChatDoctor.

What Next? 

The question for your organization is how can you leverage the lessons of an industry sector – healthcare – which is further along in the iterative implementation of AI at the Enterprise level?  We will continue to surface these lessons.  Formative questions, influenced by design and systems thinking include:

• A.I. May Someday Work Medical Miracles. For Now, It Helps Do Paperwork: What processes discussed in this NYT article re: productivity gains relative to rote, repetitive processes, apply to operational challenges within your organization?  Is there a minimum viable product (MVP)/alpha implementation of LLMs. NLMs, or NLP which you are intregrate into your stack to start measuring results?
• Pattern Recognition, Anamolies, Defects and Visual Computing:  It is not surprising that radiology and biomedical document use cases were prominent in the Hugging Face databse. X-rays and radiology have long held the position as the most oft quoted example of the impact AI will have on healthcare.  The use case here is pattern recognitiion of vast amounts of data in a visual format.  For example, if you are in the CPG or computer hardware subsectors, is there a place in your supply chain where visual elements and documents at scale can be used for quality assurance (in packaging or electronic component part inspection?  What does this MVP and alpha test look like within your organization?  
• Reframe the notion of “clinical” interaction with a customer for your organization and industry sector:  Some industry sectors call it “the customer journey”, while other more technical industries use of the discipline of human computer interaction (HCI).   Regardless, leverage systems thinking into your organization by evaluating how these key characteristics and applications of clinical language models apply to your organization.  Here is a list made generic for your evaluation: 
  • Record Processing
  • Decision Support
  • [Customer] Communication
  • [Your Industry Sector or Subsector here] Literature Summarization
  • Diagnostic Assistance
  • Natural Language Understanding
  • Translation and Multilingual Support
  • Privacy and Security
• Find your Industry Sector Community of Practice:  Does the Hugging Face community already have datasets which speak to your problem set or systems thinking? If not, where are the subject matter experts and AI Developers gathering to scale community, collaboration, and code. 

And, of course, continue to return to participate in the  OODA Loop and OODA Network communities. 

Additional Resources

Hugging Face has been included in our previous analysis of LLMs, NLMs, and NLP – including the following valuable insights and resources available to your organization as you flesh the role of open source AI in your organizations “AI for Enterprise” strategy. 

• OODA Loop 2022: The Past, Present, and Future of ChatGPT, GPT-3, OpenAI, NLMs, and NLP:  As the Tok-Tok security threat, Elon Musk’s Twitter debacle, and ChatGPT all vied for the top spot as the final big story in the technology-obsessed zeitgeist for 2022, we focused on ChatGPT in the context of OODA Loop research and analysis over the course of the year on AI, machine learning, GPT-3, neural language models (NLM), and natural language processing (NLP).
• OpenAI, Hugging Face and DEFCON 31 (August 2023) on Red Teaming ChatGPT, Large Language Models, and Neural Language Models:  We recently looked for patterns in the exponential adoption of ChatGPT, large language models (LLMs), and neural language models (NLMs)  – and how it has intersected with the red teaming discipline.
• “We Have No Moat”: Tracking the Exponential Growth of Open Source LLM Performance (Latency and Throughout):  Hugging Face has created an “Open LLM-Perf Leaderboard”  for your organization to track, quantify and evaluate this exponential growth of LLM performanceLike the Unix, Solaris, Irix, AIX Open Source operating systems that became the Linux-led open source operating system and the broader open source software movement (over a 30 to 40 year time period) the open source large language model “market” is poised to eat up market share in an exponentially fast time frame.   .
• The use of the company’s Cerebras’s open-source GPT models trained at wafer scale in some recent on-chip benchmarking:   “The trained models Cerebras has released, available under the permissive Apache 2.0 license, have been downloaded more than 200,000 times from HuggingFace at the time of writing (about two weeks after release). They are trained on the public PILE dataset from Eleuther.  Training seven models of different sizes allowed Cerebras to derive a scaling law linking the performance of the model (prediction accuracy) to the amount of compute required for training. This will allow the forecasting of model performance based on training budgets. While other companies have published scaling laws, this is the first using a public dataset, the company said.”; and
• The BigScience initiative:  A consortium led by AI company Hugging Face, made up of roughly 500 researchers (some of whom are from big tech firms). The group of volunteer researchers will be building and studying an open-source language model, which was discussed in 2021 in the context of The Current AI Innovation Hype Cycle: Large Language Models, OpenAI’s GPT-3 and DeepMind’s RETRO.
• The DALL-E of 3D Modeling: OpenAI’s Point-E May Accelerate Metaverse Production:  Text-to-Image systems like OpenAI’s DALL-E 2 and Craiyon, DeepAI, Prisma Lab’s Lensa, or HuggingFace’s Stable Diffusion, have rapidly gained popularity, notoriety, and infamy in recent years. 
• Although not mentioned directly in the leaked Google document breaking Down the Exponential Future of Open Source LLMs, Hugging Face’s open-source models will most definitely be in the mix.

AI Discipline Interdependence: There are concerns about uncontrolled AI growth, with many experts calling for robust AI governance. Both positive and negative impacts of AI need assessment. See: Using AI for Competitive Advantage in Business.

Emerging NLP Approaches: While Big Data remains vital, there’s a growing need for efficient small data analysis, especially with potential chip shortages. Cost reductions in training AI models offer promising prospects for business disruptions. Breakthroughs in unsupervised learning could be especially transformative. See: What Leaders Should Know About NLP

Technology Convergence and Market Disruption: Rapid advancements in technology are changing market dynamics and user expectations. See: Disruptive and Exponential Technologies.