Introduction To Guest

Dr. Lawrence Ngo is a practicing radiologist with PhD in neurobiology. He has received his MD PhD from Duke University School of Medicine and is now the cofounder and CEO of CoRead AI, a start-up that has developed machine learning algorithms that analyzes the images and reports from radiology exams, and flags studies with high probability of error for peer review. They are essentially looking for blind spots and things that could have been missed by a radiologist and flagging them.

Dr. Ngo has always been fascinated by how to bring in advanced technologies to improve the human condition. Throughout his life, he has always jumped back and forth between highly technical fields and humanities. For example, he majored in philosophy and then went to medical school. Dr. Ngo soon went on to receive his PhD in neurobiology, creating a project that used MRI scanners to scan people’s brains while they make moral decisions, and really understanding how their brains work for that. Dr. Ngo has always had an interest in combining the two.

The latest thread he’s been working on is from several years ago, when IBM created an algorithm called Watson that was able to play Jeopardy against reigning Jeopardy champions. By doing so and playing those games, and being able to beat the top champions, Watson was able to prove that it was able to do something that people thought was uniquely human. However, what really caught Dr. Ngo’s attention was when the headlines claimed that Watson was going to medical school. Watson’s algorithm was being trained to take the same test and try to beat human’s doing so. This together was how Dr. Ngo initially became interested in really trying to figure out how AI could be applied to the field.

Fast forward to when he became a resident. After medical school, Dr. Ngo got more into clinical practices. He got connected to a large teleradiology company that had access to very large amounts of data. Dr. Ngo had developed some technical skills during his PhD in data science using some machine learning and decided to join their incubator program. This was tough since he was in the middle of his residency.

Coming Up with The Idea

For Dr. Ngo, it was the confluence of two different threads radiology resident in the daytime and training machine learning algorithms radiology at night. On one hand, he was a radiology resident. The time it takes getting your residency also comes with a lot of mistakes. He had to become familiar with his limitations. On the other hand, Dr. Ngo oversaw doing diagnosis and interpretation of image imaging. And at nights, for example, when no supervising physician was around, Dr. Ngo would learn of the things he could have done better the morning after. He was trying to train algorithms using AI. Dr. Ngo also had an advantage since he was a radiologist himself. He spent a lot of time reviewing the outputs of the AI and trying to label the data where the algorithm had made mistakes. In Dr. Ngo’s findings, he saw that at times algorithm was correct – and the initial radiologist who was making the diagnosis was making the mistake.

To better understand how this works, Dr. Ngo had an algorithm that looks at images like an X ray or CT images. Then another algorithm looking at the report that radiologists have rendered during regular clinical practices interpreting the images.

As Dr. Ngo and his team continued to develop more and their algorithms got better, they started finding more mistakes from radiologists. During a residency you have a safety net, a supervising radiologist checking your work. But in regular practice, you don’t have anyone going behind you to check your work. By having an algorithm that can process 10s of 1000s of images in a very small amount of time, you’re able to do a rough review of a very large number of studies, finding weaknesses in people’s practice patterns.

Production and Partnerships

CoRead AI has been very fortunate to get connected with a lot of other companies in the field. Those companies have done a lot of work in trying to build platforms at hospital systems and radiology practices to deploy AI algorithms. The field of Radiology and AI is very hot with a lot of investments. Dr. Ngo believes that it’s important to establish platform at a network of hospitals around the world where any other company can plug in their AI algorithm for whatever use case and then be able to serve on the platform.

CoRead AI’s focus is narrower. They have the medical expertise and the AI development expertise. They want to develop some high-quality models that both look at the images and the reports and are happy to partner with as many people as possible to get that delivered to different hospitals and radiology practices. CoRead.AI is open to as much collaboration as possible.

Challenges Encountered

There are always challenges, as Dr. Ngo explains. The first thing is with AI development, the quality of the data is super important. They deal with very large datasets. Even though Dr. Ngo is highly motivated to make sure that annotations are as high quality as possible, mistakes happen along the way in terms of labeling data. Many people focus on the out-algorithm development and fine tuning everything, but it comes down to really understanding the data and making sure that it’s labeled in a high-quality fashion has been a challenge.

The other challenge in the field of algorithm development is not as rigorous regarding controlling for biases. It’s difficult to run studies showing that algorithms can impact how the patient outcomes and those studies are starting to come out now. There’s a struggle in trying to make sure that ultimately, you’re able to prove that the algorithms are doing something important, rather than just being part of the AI hype and having an algorithm like everyone else.

Future Plans

There are two sides of the coin for radiology: improving the quality of the care that’s being provided and improving the efficiency of radiologists. The progress of CoRead AI is that they continue to improve their algorithms to a degree of higher accuracy, such that the peer review system can be more streamlined. Perhaps get to a point where the algorithms can augment the workflow. Right now, CoRead AI is helping the quality standpoint, but potentially help radiologists become more efficient as well. To augment the work that radiologists already do and help to improve higher quality care while keeping the radiologists more balanced and less partout.

Challenges Encountered

Dr. Ngo’s advice is if you’re already in the healthcare space, it’s a huge opportunity to learn machine learning and AI, because you have this advantage of knowing something about the healthcare space already. But if you’re coming from the outside, really take some time to dig deep into understanding what the flow is, how healthcare interacts with patients for that condition, how they improve outcomes for that condition, and then how economics works around that.

Also, make sure you have a strong support network around you. This has greatly helped Dr. Ngo.