AI in Santià: Transforming an Idea into a Functional AI System in Health care

What are the steps to go from an idea of an AI system to one that you can actually use? In this article we will dig deep into the development pipeline for an AI system and discuss some potential challenges you might encounter.

Throughout this article, you’ll be working on a project.

From Idea to Project

Transforming an idea into a functional AI system presents numerous challenges. The journey from concept to practical application involves:

understanding the problem,
acquiring and preparing the right data,
model development and
rigorous testing.

Beyond technical hurdles, real-world constraints such as data availability,
computational resources and ethical considerations play a critical role.

You will conceive of and pitch a novel health care product or service that leverages AI to solve an unmet need in health care.

The project should identify the core components needed to realize the
proposed AI solution and consider the following items:

The unmet need and target market for your product. Who are the likely beneficiaries of your system and what health care problem is it solving?
The overall structure of the AI system you plan to create. For example, if it’s a prediction model, what is the appropriate
prediction target and what are the relevant input features?
The data you would need to build the model and any potential data-related issues such as sample size, bias, privacy or HIPAA concerns and label collection challenges.
An implementation strategy that addresses integration with and/or modification of the target clinical or health care workflow.
Key stakeholders for your solution. How does your solution create value for them and what concerns or barriers might they raise? For instance, health care workers who may use the system, patient populations affected by decisions of your system, payers, etc.
The potential regulatory implications of your system.

The journey is marked by three pivotal stages. Training where the model is developed, validation which tests the model’s reliability and generalizability
in real-world scenarios and deployment, where the model is put into practice to solve real problems:

Stage one, training the model.

This is the stage where you actually create the AI model
you would like to use to solve a specific problem. Key elements in this phase includes data collection and preparation. The quality and quantity of data will directly determine the model’s ultimate effectiveness.

Training involves feeding this data into the model to learn patterns and relationships. During this stage, only preliminary accuracy readouts are available and provide only limited insight into the model’s performance characteristics. However, these initial metrics are just the beginning as they reflect the model’s ability to learn from the training data.

The term “Training” refers to the process of teaching an AI model to make
predictions or decisions by feeding it with training data, i.e., features and labels. The model learns to associate specific inputs or features, with specific outputs or labels.

This process involves adjusting internal parameters of the model so we can accurately map new inputs to new outputs. Training continues until the model achieves a satisfactory level of accuracy on the training data.

Data set

This process will often rely on historical, retrospective data that your
institution or organization has on hand. It’s important to keep in mind that this kind of data was not initially collected for AI development. Most of this initial data becomes part of the training set and is used to train the model, with a smaller portion is allocated as a validation set.

Do I have a sufficient amount of historical data?
it’s often possible to do a small feasibility study to see how much data you might need or alternatively, how easy it is for you to acquire raw data, potentially within the same system you intend to develop.

Do I have the right kind of data?
The nature of data plays a significant role in the effectiveness of deep learning and not all data are created equal. Suitable data types include images, text, time series and sequencing data.

What kind of data?
However, tabular data, which consists of structured data and tables like you’d find in an Excel spreadsheet, don’t work as well with deep learning. It’s not that you can’t use deep learning with tabular data, but don’t expect it to work as well as traditional prediction models.

This is because deep learning excels in identifying patterns in
unstructured data, where relationships are not predefined as in tabular formats. Deep learning does best with certain types of data, like images, because of its ability to leverage the inherent structure in the data. In images, for instance, the correlation between nearby pixels provides rich contextual information that deep learning algorithms can exploit.

This structural characteristic allows the model to discern patterns and features with a high degree of accuracy. Similarly, in text and sequencing data and time series data, there are patterns and sequences that deep learning models can recognize and learn from. Moreover, these types of data are often direct measures of the phenomena they represent. For example, an X-ray image directly visualizes the physical condition of the lungs, making a potent tool for diagnosing pneumonia. This tool direct representation often results in a higher signal-to-noise ratio compared to more abstract or processed forms of data like those in tabular formats.

In essence, the more direct and unprocessed the data, the more effective deep learning can be, as it can more readily identify and learn from the underlying patterns and signals inherent in the data.

Features

In AI, features are individual, measurable properties or characteristics of the
phenomenon being observed. Think of them as the raw data that feeds into an AI model.

For example, in medical imaging, features could be specific pixel values or textures in an image. In a language model, they might be the individual words or phrases in a text. Features form the basis of what the model learns from.

Label

Labels contain the correct answer for a given set of features. In essence, what most AI models are doing is trying to predict the correct label on the basis of the features.

While we’re on the importance of good data, it’s time to reiterate just how important the labels are.
Acquiring accurate labels is often challenging, expensive and time consuming. In medical context, these labels are usually provided by health care professionals, a process that can be lengthy and subject to variability as different doctors might have differing opinions. The quality of your labels will set the limit for your system’s accuracy.

For example, in a medical application of AI, the labels could be the presence or absence of a disease in an image, and the features would be the pixels of the image itself. Another example of a label would be whether or not a patient is readmitted to the ICU within 30 days of discharge. The features in this case could be the elements from the patient’s record, including labs, vital signs and procedures they received during their ICU stay.

Evaluating the performances
Evaluating the performance of your model during the stage relies on proxy metrics. As direct outcomes across metrics are typically unmeasurable until stages two or three of the pipeline. Common metrics include classification accuracy, sensitivity and specificity, the area under the receiver operator curve or ROC. These metrics are context dependent with no universally good
values. It’s important to note that your model may fail for a variety of reasons at this stage.

Failure mode

A particularly concerning failure mode is when a model appears effective but is subtly overfit or confounded, making these flaws hard to detect at stage one. An illustrative example of this failure mode is in developing an AI system for diabetic retinopathy screening. Suppose you train a model using retinal imaging from all patients who are diagnosed with diabetic retinopathy from your institution’s EHR.

However, upon inspection, you notice that the model has learned that the presence of laser scars, which is a treatment for diabetic retinopathy, is highly predictive of a patient having the condition. Unfortunately, this association will likely reduce its ability to serve as a screening tool, since patients you are screening will not yet have been treated and thus will not have laser scars. This scenario highlights the importance of training the model on data that reflects your intended real world use case. In this instance, the model should only be trained on data from treatment-naïve patients. As you can see, the first stage of developing an AI system is a crucial juncture where
careful planning, thoughtful data selection and a clear understanding of the capabilities and limitations of deep learning are essential. It’s important to recognize that not just the quantity, but the quality and type of your data significantly influence the effectiveness of your model. Deep learning, while powerful, is not a universal solution and requires appropriate data to learn correct prediction procedures. This stage also reinforces the importance of high quality labeling, as it directly impacts the accuracy and reliability of your AI system. Evaluating your models performance through proxy metrics provides an initial understanding of its effectiveness, but be aware of the
potential pitfalls such as overfitting or data biases which can lead to misleading results.

Predictions

Features and labels are the building blocks of AI training, guiding the model to learn and make predictions.

The training process itself is always a balancing act where the goal is to
achieve not just high accuracy on the training data, but also effective generalization to new data.

Start the training model

Before you start to train a model, there are several important
questions you should ask yourself.

First, you should ask yourself whether prediction or automation capabilities will improve your organization’s capacities. With perhaps the exception of some LLM applications, what an AI system will do is give you a prediction for a very specific outcome.

Stage two, validation of the model.

The gold standard for model validation is prospective validation where the model is tested under controlled real-world conditions. This stage is crucial for assessing the model’s true efficacy and generalizability. Prospective validation involves deploying the model in a realistic setting using new data that it hasn’t encountered during training. This step is critical in health care AI where the model’s predictions can have significant implications for patients in real-world settings may not always match the conditions under which the model was trained.

In the context of AI and overfitting, tuning and validation data play crucial role roles in developing robust models.

In stage two, the focus shifts to validating the model developed in stage one using prospective data. At this stage, test the model in a live environment, often designed like a trial with preregistered endpoints and conducted over a set period of time. Much like traditional software, the model is frozen during this phase. It does not continue to learn or update. This constraint is crucial to ensure that any performance observed is due to the model’s existing capabilities, rather than incremental learning or modifications made during the validation period. The goal is to demonstrate that the model functions effectively in real-world conditions. This stage is where you verify the model’s reliability and accuracy with incoming data which hasn’t been seen by the model during training.
You also have the first opportunity to assess metrics that were difficult to assess during stage one. For example, you can measure how long it takes a physician to process a case with access to your AI tool, with the goal that it will cut down time to diagnosis.

This would be something that would be hard or impossible to measure during the artificial setup of stage one. However, a model may fail during perspective validation for several reasons. For example, when the validation includes data from new hospitals or diverse populations not well represented in the retrospective data of stage one, issues may arise. For example, if you straddle model performance by ethnicity, it could reveal that the model’s performance vary significantly across different groups, indicating a lack of generalizability. This could be due to several factors such as biases in the training data, overfitting to the features present in the original data set or a mismatch between the training and validation environments. Another failure mode
could be that integration of your tool into a real world workflow is difficult and results in an overall deterioration of important outcomes. Note, most AI studies to date have not undergone such prospective validation and according to a 2019 study, only a small fraction of studies have reached this stage. This implies that most of the studies you have likely read about are stage one studies, and we do not have a good understanding of their real-world performance characteristics. This gap underlies the necessity for a comprehensive testing.

Without this step, there’s a risk that the model will not perform as expected in the real-world, which can have significant consequences, especially in high stakes fields like health care.

Overfitting

Overfitting occurs when a model memorizes the training data so thoroughly that it loses the ability to generate new insights or discern errors from the foundational model. It becomes overly specialized in the training set, thus failing to generalize effectively when presented with new or varied data.

Techniques like cross validation, regularization and using simpler models are employed to mitigate the effects of overfitting, aiming to ensure that the model not only performs well on the training data, but also generalizes effectively to new data.

Overfitting is a common pitfall in this journey, highlighting the importance of
carefully tuning the model and evaluating it with validation data.

Tuning data

Tuning data is used to optimize the model after the initial training phase. This process involves adjusting the model’s parameters to balance between learning the training data and maintaining the ability to generalize to new data. It’s a delegate act of ensuring the model doesn’t just memorize the training data, i.e., overfit, but learns to make accurate predictions on data it hasn’t seen.

Validation data

Validation data, on the other hand,
is a separate data set not used in training or tuning. It serves as a check against
overfitting by providing an unbiased evaluation of the model’s performance. By
applying the model to validation data, developers can assess how well the model
generalizes to new unseen data. This step is crucial in determining if the model has learned the underlying patterns or if it’s merely memorizing the training data.

Effective use of tuning and validation data is key to developing AI models that are reliable andperform well in real-world scenarios beyond the confines of the training data set.

Stage three, the final stage is deployment.

This phase involves integrating the AI model into actual use cases or workflows. Deployment is a significant milestone but not the end of the journey. Continuous monitoring is necessary to ensure the model maintains its accuracy and effectiveness over time. Moreover, post-development, the
model may need update or retraining to adapt new data or changing environments.

Stage three has likely been your ultimate goal. Your model is now deployed in a real health care setting is helping make decisions. Congrats, well, maybe. Things can still go wrong even at this stage. Deploying an AI system within a health care environment is a complex process that requires planning and consideration of numerous factors.
Key considerations include the integration of the AI system with the existing health care IT, such as electronics health care records, ensuring the system complies with patient privacy laws and regulations and the ability of the AI system to complement the dynamic nature of medical data and practices. Once again, however, there can be problems at this stage, even for AI systems that pass stages one and two with flying colors. For example, consider the scenario where an AI system, though previously validated for accuracy, experiences a significant drop, as revealed during an audit.
One potential cause for this could be changes in the health care environment, such as changes in the way data is coded in the EHR. A notable example of this happened in 2015 when billing changed versions from the ICD-9 to ICD-10 format.

If the AI system was trained on data from the old EHR system and heavily relied on the specific format and structure, the transition to a new version could lead to a mismatch between the expected and actual data format, rendering the AI system ineffective or even useless.
This scenario is an example of a problem known as data set shift, a phenomenon where the statistical properties of the data the model was trained on differ from the properties of the data it encounters in deployment. Data set shift can occur due to various factors such as changes in population demographics, evolution of clinical practices or alterations in data collection methods.

It may not be obvious, but nearly all AI systems have a very strong assumption at their core. This assumption is that the world is unchanging and that what happened yesterday is a very good predictor for what will happen tomorrow. So, when the world actually does change, almost all AI systems will not be able to cope, and all the performance metrics you measured will likely be invalid.

To mitigate these kinds of failures, you must constantly audit and monitor the performance of your deployed system, so that you can quickly catch and correct this kind of failure. Deploying an AI system in health care setting marks a significant milestone, but it also begins a phase of ongoing vigilance and adaptation. As the system encounters the dynamic nature of real-world data, challenges like data set shift must be anticipated and managed.

The ultimate goal – Conclusion

The ultimate goal is a system that not only performs well under controlled
conditions but can also survive the dynamic, unpredictable world of health care.