The five-step guide to AI adoption in clinical practice

Jeroen van Duffelen proposes a five-step programme for adoption of artificial intelligence in clinical practice.

Adoption of medical imaging AI is about getting your hospital or screening programme ready to implement the right solution for a clinical need. Running into speed bumps along the way is common for early adopters. How do you define the needs, budget, and outcomes? Which boxes should you check when selecting vendors? How do you manage internal stakeholders? The adoption curve is steep. Luckily, you don’t have to climb it alone.

Drawing from our experience deploying AI in clinical practice and lung cancer screening, I’ve designed a five-step guide to streamlined adoption. If you’re looking to adopt artificial intelligence but don’t know where to start, these actionable tips and advice will see you through. For the video breakdown of the steps, watch this presentation from ECR 2020.

1. Consider

Where do you start working with AI? First, look away from all the solutions out there, and focus on your organisation. Bring together all the stakeholders into a project team that includes the sponsor, if applicable, IT and legal representatives. Involving them from the beginning will expedite the process.

Start with defining the challenge you are looking to solve, or the specific clinical question that is relevant to your workflow. Some hospitals are looking to experiment with the technology, while others aim to solve a particular issue. Over the past years, I have seen the latter getting more out of AI, which is why my advice is to start from a clinical challenge.

When considering this challenge, make sure to already determine your expected outcomes. When is the adoption a success? Are you aiming to have an AI solution in use? Should it apply to a certain patient population, or yield specific results like time or cost savings?

Also, although it may seem early, this is also the stage to organise a budget dedicated to the AI solution. The size of this budget should relate to the cost or time savings a solution is expected to bring. Both the amount secured and its source will impact the next steps. For example, it will guide you to look for PhD researchers versus seeking a vendor that offers a mature solution.

2. Evaluate

The AI in healthcare space is widely populated; a Google search or a look at the list of vendors at the RSNA can confirm that. To weigh the existing options for your scope, do your (desk) research using this high-level checklist for each solution:

How was the AI solution validated?

It is important that the claim that the AI solution has validated covers the use case you identified in the previous step. Take the time to understand if the manufacturer has done studies confirming this claim.

How does it integrate into the workflow?

Try to get a feel of the amount of effort needed to add an AI system into your workflow. A good practice is to start with an AI solution that is easy to integrate with the current workflow and IT infrastructure. Workflow integration is of utmost importance for the radiologist; in this article, we explained why that is and how it works.

What regulations does the solution fulfill for use in clinical practice?

Commercialising medical devices requires a CE Mark in the EU and an FDA clearance in the US. Note that local regulations may apply to different countries. Again, pay attention to which claim is covered by the acquired certification.

3. Choose

By this stage, you should have narrowed your search down to a few vendors. This is the moment to go in-depth into the workflow and test if a specific solution is a good fit from both a clinical and a technical standpoint. A well-integrated AI system should not create hurdles for physicians, such as requiring them to leave their workstation to upload studies. It should further blend within the existing IT infrastructure.

There are two checks that are vital to make the right choice:

Validate the accuracy

Legitimate vendors would have done a study and can provide a clinical background for accuracy. To know if the solution is good at performing the defined task for your organisation, ask questions about the datasets used to develop and test the AI solution.

There are three datasets required to build an AI model: a training dataset, a validation dataset, and a test dataset.

The test dataset is the most relevant to look at because it is what the accuracy is based on. The performance on this dataset should be applicable to your hospital, with its specific protocols, type or number of scanners, and patients. To achieve this, the test set must cover the patient population your organisation serves (e.g. types of patients, comorbidities distribution, etc.). Thus, inquire about the specifics of the test set and the performance of the AI model on this dataset.

Secondly, you may want to know what the size of the training dataset is and how it was labelled. Both quantity and quality are important to train an accurate AI model. Labeling the data should be done by experienced radiologists, preferably with multiple readers per study.

Check the regulatory compliance

In Europe, medical device classification is divided up between risk Class I, Class IIa/b, or Class III. If looking for a solution for clinical practice, be wary of Class I medical devices. The new Medical Device Regulation, which will come into force in May 2021, will require many AI products currently classified as Class I devices to update their classification. For instance, software that supports diagnostic decisions should fall under Class II at a minimum. For more guidance on the new regulation, read our recent expert piece.

Apart from the regulatory approval, check if the vendor also has a quality management certification (e.g. ISO 13485). Reviewing the data processing policy and the cybersecurity measures in place will further help you understand if the AI company is going the extra mile in regard to safety.

A bonus tip for the choosing stage: do a reference check. Ask other organizations how they are working with the AI solution you have chosen. You may get the insights you need to make the final decision.

4. Approve

Approving the chosen solution internally requires the involvement of and coordination between IT and PACS administrators, procurement officers, physicians, often also privacy departments and legal officers. If you have a project team in place since the first step, you should be well on track.

To move forward and avoid delays, assign an internal AI champion responsible for driving the project. This may be an executive sponsor, a budget holder, or a department manager. One of my learnings from past deployments is that the risk of failure is high without a person fulfilling this role. What I have further learned as vendors is the importance of empowering the AI champion, by providing the necessary information and documentation in a timely manner.

Furthermore, make sure end users are trained to use the new medical device. If they don’t benefit from it, the impact of the AI solution will be limited. Additionally, setting up a feedback mechanism with the AI vendor from the get-go will help improve the AI product.

5. Deploy (& evaluate)

All the paperwork is signed – well done! To make the deployment work, create a clear project plan, including actions, timelines, and owners. Depending on the type of deployment – on-premise or cloud-based – different actions will be needed. As outcomes, set the deployment and acceptance dates, make agreements on the service levels, fixes, and upgrades, and discuss post-market surveillance.

The initial or trial phase of using the AI solution should show if it answers the problem you were trying to solve. It is a good moment to revisit step one and start evaluating the results to decide if you will continue using the solution.

A common question I get at this stage is: “Do I need to do a full clinical study?” The answer fully depends on the purpose of using the product. It is necessary for research, but not for other use cases. What matters is validating that the AI solution is adding value to your clinicians and their patients.

Make it better

AI adoption does not end with deployment. Service and maintenance are essential, and their quality often a differentiating factor between AI vendors. The implementation process usually acts as a good test for the AI companies fulfilling their promises and being prompt when handling requests.

Beyond these five steps, you and your organisation play a role in improving the chosen AI solution through valuable feedback and feature suggestions. The collaboration between humans and software allows us to achieve much more than humans would on their own. If done right, it can be transformative for patients.

Are you ready to start the AI journey? Get in touch!

Jeroen van Duffelen, COO & Co-Founder

Jeroen van Duffelen is COO and co-founder of Aidence. Jeroen’s entrepreneurial spirit led him to teaching himself software engineering and starting his own company commercialising an online education platform. He then tried his hand in the US startup ecosystem where he joined a rapidly scaling cloud company. Jeroen returned to Amsterdam where he ran a high-tech incubator for academic research institutes, it is here Jeroen first got his taste for applying AI to healthcare.

Is artificial intelligence the key to effective and sustainable lung cancer screening?

Lizzie Barclay doctor

Dr Lizzie Barclay explores how artificial intelligence can influence lung cancer screening.

Radiology as the starting point

Imaging plays a fundamental role in lung cancer screening programmes. So, when it comes to improving technology to support the programmes, the radiology department is a good place to start.

The goal of screening is to pick up early cancers which can be treated and potentially cured, therefore improving patient outcomes (as outlined in the NHSE long term plan). Low dose CT has been shown to provide sufficient image quality for detection of early disease, whilst minimising radiation dose in asymptomatic individuals. Thoracic radiology expertise is required to determine which lung nodules may be malignant and therefore require invasive investigation, and which are likely benign and can be monitored with intermittent imaging. Appropriate follow-up recommendation helps avoid unnecessary invasive procedures, such as biopsies, and minimise patient anxiety, which are important measures of the efficacy of lung cancer screening programmes.

End to end lung cancer screening involves input from many healthcare professionals, and intelligent computer systems across specialities would benefit multidisciplinary teamwork. Thus, beyond image analysis, there are many opportunities for technology to add further support for effective and sustainable screening programmes. For instance, it could aid in the optimisation of image acquisition, access to imaging reports and relevant clinical details, tracking patient follow up, or in communication between patients and GPs.

Where AI-based image analysis makes a difference

Reading and reporting CT scans is time-consuming, and within a workforce which is already under strain, introducing a new CT-screening programme seems like a tall order. AI-driven solutions can support radiologists and contribute to successful lung cancer screening by bringing improvements in three areas:

  1. Performance

Computer intelligence can increase the performance and productivity of CT reporting, freeing up time for radiologists to spend on clinical decision making and complex cases. Specifically, AI software is well-suited for precise:

  • Detection of elusive lung nodules, and differentiation of subtle changes
  • Automatic volume measurements, to help determine the appropriate frequency of monitoring (e.g. stable vs growing nodule, according to the BTS guidelines).

What further distinguishes computers from humans is the absolute consistency in their high performance, without being impacted by common external stressors to which a radiologist would be exposed (e.g. time-pressure, workload and interruptions).

  1. (E)quality

Having a ‘second pair of eyes’ looking at the scan can increase the confidence of the radiologist in their own assessment. Additionally, making the AI-driven, accurate measurements available regardless of the level of expertise of the reporting radiologist could not only benefit quality assurance, but also equality within the radiology department. The use of AI would reduce the need for all scans to be reported by the most experienced thoracic radiologists with interest in early lung cancer detection, and instead facilitate spreading the workload across the workforce.

Another use case concerns quality assurance when outsourcing to teleradiology companies. AI-based image analysis can improve consistency of reporting, drive the recommended terminology use, and, essential for lung cancer screening, ensure access to relevant prior imaging for comparison and change assessment over time.

  1. Efficiency (via integration)

An intelligent computer system should not slow down reporting turnaround times, but improve efficiency, as well as quality, to ultimately minimize time to diagnosis (for example, the NHSE long term plan introduces a 28-days standard from referral to diagnosis or rule out).

Older CAD technology was often described as ‘clunky’ – requiring images to be uploaded to separate systems for analysis. Additional manual steps between image acquisition and the radiology report make the process time consuming, and often require radiology support staff to manage the workflow. It is important to consider allocative and technical efficiency which play important roles in the evaluation of screening programmes, and their impact on healthcare systems.

An AI-driven image analysis software which is fully-integrated in the radiologist’s pre-existing workflow can provide automatic results, without needing additional departmental resources. An additional benefit of fully-integrated AI solutions is that their use is not restricted by time or place, therefore supporting flexible and remote working. In the context of the COVID-19 pandemic, it’s been encouraging to see the increase in remote reporting, whilst maintaining a functioning department, in many hospital trusts. Going forward, it will be interesting to see whether radiologists will have the option to continue to work remotely where possible.

Valuing input from healthcare professionals

New lung cancer screening programmes will be monitored regularly to evaluate their effectiveness and determine areas for review. Commitment from all parties to work together will facilitate optimisation of the pathway to achieve better patient outcomes and positive impacts on healthcare systems.

In our experience, close collaboration between medtech and healthcare professionals is important for learning lessons along the way. Understanding radiologists’ needs helps tech teams develop a clinically valuable tool.

For example, Aidence’s interactive lung nodule reporting tool, Veye Reporting, was designed based on the needs of radiologists involved in reporting lung screening scans. From our conversations with them, we understood that following the detailed and complex reporting protocols in lung cancer screening programmes make for labour-intensive, repetitive tasks.

Veye reporting

To help them produce reports that follow the standardised NHSE proforma and facilitate audit for quality assurance, we added Veye Reporting as a feature to Veye Chest, focusing on making it easy-to-use and efficient. With this tool, the radiologists further have control over which nodules to include in the report, different sharing options, and the choice to add incidental findings.

What’s next?

Cancer services have been impacted by the COVID-19 health emergency. In the UK, screening has been paused and planning to (re-) start at the end of 2020 or beginning of 2021. Talks of introducing screening are ongoing in various European countries, as are concerns of catching up with the backlog of screening scans.

The British Society of Thoracic Imaging and the Royal College of Radiologists released these considerations for optimum lung cancer screening roll-out over the next five years. Their statement below is a reminder of why it is worth overcoming challenges and leveraging technology to make screening programmes a success:

BSTI_RCR statement

Dr Lizzie Barclay, Medical Director

Dr Lizzie Barclay’s areas of interest are thoracic radiology and medicine, innovation, and improving patient outcomes and healthcare professionals’ wellbeing.

Lizzie is originally from Manchester, UK. After graduating from the University of Leeds Medical School (MBChB), and Barts and the London School of Medicine (BSc sports & exercise medicine), Lizzie spent four years working as a doctor in Manchester and Liverpool NHS Trusts, including two years in Clinical Radiology. She has presented her work on lung cancer imaging at national/international conferences, and recently contributed to Lung Cancer Europe’s “Early Diagnosis and Screening” event at the EU Parliament in Brussels.

https://www.aidence.com/

You may be interested in the BIR Lung Cancer Imaging: Update for the not-so-new normalon 11 September 2020. This will be available for members in the BIR online learning libraryafter the live virtual event.

 

“The Radiology Reset Button – overcoming the normalcy bias”

Fodi KyriakosFodi Kyriakos explores how the COVID-19 pandemic could be the catalyst for change in radiology and encourages our community to grasp the opportunity to “seize the moment”  and plan for recovery.

At the beginning of 2020, if someone had told radiology leaders that all NHS outstanding reporting backlogs would be reduced to virtually zero by May, I’m sure they would have looked at you in disbelief and asked what sorcery had been involved, but this situation is exactly where we find ourselves today.

 

Normalcy Bias – Noun [edit]

normalcy bias (plural normalcy biases)

The phenomenon of disbelieving one’s situation when faced with grave and imminent danger and/or catastrophe. As in over focusing on the actual phenomenon instead of taking evasive action, a state of paralysis.

Historical challenges

In the past, it has often taken lots of effort to either invoke or accept change of any kind in radiology and for those managing services, there’s also been a certain amount of risk associated with putting your head above the parapet or being a trailblazer. It has been sometimes easier to follow the well-trodden path rather than to create a new one. Workloads and budgetary constraints have also been a disabler, restricting decision making to the ‘here and now’. This has resulted in failing, or in most cases, not being able to foresee or plan for events that have never happened before, such as an event like a pandemic crisis. Psychology refers to this state of being as normalcy bias. For those who are not familiar with the term, you will certainly be aware of its connotations and radiology now finds itself at this cross-roads.

Ever since the introduction of digital radiography and PACS, NHS radiology reporting backlogs have been a contentious issue among experts, and a recurring feature in the mainstream media! Often being highlighted (and with some justification) in relation to areas such as missed cancer diagnosis, where even the slightest of delays can have a significant bearing on the overall outcome.

Serious backlogs

The extent to which backlogs were a serious issue in the UK was further exacerbated by various Care Quality Commission (CQC)  inspections, which raised concerns regarding reporting backlogs that resulted in delayed or missed diagnosis of conditions that may have otherwise been picked up.

By the end of February 2020, the situation of backlogs was as much an issue as at any time before. Insufficient reporting capacity had led to a build-up of outstanding reports, which in turn meant that outsourcing was at its highest ever levels and growing pressures to meet new deadlines, such as the cancer pathway targets, were increasingly exposing the lack of options available to resolve the problem.

So, you would have been excused if you thought that a crisis such as the COVID-19 pandemic would simply exacerbate the reporting challenges facing radiology. However, this has not been the case. Instead, we have witnessed radiology’s own “clear the decks” exercise, where in fact the complete opposite situation has occurred, resulting in backlogs across the UK being virtually eliminated. Who would have thought that the worst crisis to hit the country (and the world) in 75 years would be a catalyst for NHS radiology departments to press the reset button?

Reset image

Of course, we recognise the superficial nature of this situation. During the pandemic, practically all routine referral activity came to a grinding halt, which allowed radiology to concentrate on COVID-19 and Emergency Department (ED) patients. Chest X-rays and CTs were identified as two of the key diagnostic tools for the virus, but the volumes were manageable. Accident and Emergency footfall was reduced to almost 50% of its usual figures, so reporters were practically able to deliver a ‘Hot-Reporting’ examination for every patient requiring imaging. Something which ED and Intensive care unit (ICU) consultants have grown quickly accustomed to.

During this time, radiology was also still required to work to critical staffing levels, so radiographers and radiologists were covering 24/7 rotas, but due to the lack of activity outside of portable X-ray scanning in ICU, many staff were not being utilised. So, while this enabled the catch up in radiology reporting to take place, what we witnessed was the ‘ying and yang’ of radiology. On the one hand, integral to the continuity of a patient’s pathway and critical to defining an outcome – AND on the other hand, completely dependent on throughput from referrers to maintain activity levels.

Seizing the moment!

So what happens next? Well, in a world where we can guarantee almost nothing, in this situation, we can guarantee that radiology will remain the centre point for the recovery phase of the pandemic, but with the added challenge of complying to ‘social distancing’ and ‘equipment cleaning’ guidelines, how do we manage the continuation of treating COVID-19 patients, while reintroducing ‘business as usual’ and ‘deferred’ patients whose treatment has been delayed?

The “Reset Button” has enabled something else to happen. For the first time, there is now some headspace to plan for the recovery phase and for the next phase at least, there is now funding available to support the recovery. So how do we avoid going back to where we were before the pandemic? How do we seize the moment?

Time to make the changes!

Albert Einstein once famously said: “We can’t solve problems by using the same kind of thinking we used when we created them.” This quote has never been more poignant in the present day and while the pressure to manage change will be at its highest, this is the right time to make these changes happen! With the benefit of ‘The Reset Button’, if we can learn from the past and apply new ways of working moving forward, we can avoid falling into the trap of the normalcy bias and witness the radiology reset button offering a new, efficient and more streamlined radiology department moving forward.

Everything you wanted to know about radiology but were afraid to ask…

On Wednesday 17 June, a live event organised by InHealth, in partnership with The British Institute of Radiology and the Society of Radiographers is taking place, titled: “The Radiology reset button has been pressed”. The aim is to tackle these challenges and support radiology managers as they enter the recovery phase. It will bring together senior figures from radiology and within healthcare to offer insights, opinions and advice on how we can approach this coming period and use what positives we have experienced during the pandemic to create service improvements throughout radiology.

There will be opportunities for radiology managers, clinical leads, radiographers and radiologists to put their questions to the speakers in the panel discussions after their presentations.

REGISTER FOR THE RADIOLOGY RESET BUTTON HAS BEEN PRESSED HERE

(The event is free for all)
About Fodi Kyriakos

Mr Fodi Kyriakos is a former director of RIG Healthcare and founder of RIG Reporting,
the UK’s first provider of external radiographer reporting services. In 2016 he joined The InHealth Group following its acquisition of RIG Reporting and is now the Head of Reporting across the Group. His service specialises in delivering plain film reporting solutions and is the only provider to offer both on-site and telereporting services.
Fodi has over 22 years experience in workforce and staffing solutions and 17 years working exclusive within Imaging and Oncology. He is a member of the Institute of Healthcare Managers and a regular contributor of professional development events across radiology.

 

Bringing together Science, Faith and Cancer Care

Slide2

The Revd. Canon Dr. Mike Kirby, Chair of the BIR Oncology and Radiotherapy Special Interest Group, has a wealth of experience as a senior radiotherapy physicist, working on national guidance, developing clinical practice and teaching radiography students. As if this doesn’t keep him busy enough he has also taken on the role of Canon Scientist at Liverpool Cathedral where he is working to encourage dialogue and discussion about science and faith. Here he explains what the role involves.

I began work in the UK’s National Health Service more than 30 years ago, as a Radiotherapy Physicist at the Christie Hospital, Manchester UK.  Alongside my routine clinical work, my main research interest was in electronic portal imaging and portal dosimetry.  I then helped set up Rosemere Cancer Centre in Preston, UK from 1996 as deputy Head of Radiotherapy Physics and Consultant Clinical Scientist there.  During that time I contributed to and edited national guidance documents such as IPEM Reports 92, 93 and 94 and the multidisciplinary work, ‘On-target’.

My work moved back to the Christie in 2007 and as Head of Radiotherapy Physics and Consultant Clinical Scientist for the Satellite Centres, I helped to lead their development in Oldham and Salford as part of the Christie Network. My research and development work has primarily focused on electronic portal imaging, developing clinical practice and equipment development.

Mike Kirby4

More recently my focus has been on teaching and learning for radiotherapy education as a lecturer (Radiotherapy Physics), especially using VERT, for Radiotherapy programmes in the School of Health Sciences, Liverpool University; but always with a focus on the wider picture of radiotherapy development having served on both IPEM and BIR committees throughout my whole professional career.

 

Alongside my scientific work, I am a priest in the Church of England; having trained and studied at Westcott House and the Universities of Cambridge and Cumbria, I hold graduate and postgraduate degrees in Theology.

Mike Kirby

My ministry has mainly been in the Cathedrals of Blackburn, Chester, and Liverpool (Anglican) where I was Cathedral Chaplain.  I have recently (Feb 2020) become a Residentiary Canon of Liverpool Cathedral, with the title of Canon Scientist the primary aim of which is to encourage dialogue and discussion about science and faith.

 I am a member of the Society of Ordained Scientists and have given numerous talks on Science and Faith to schools, colleges, churches and other institutions.  These have included organising lecture series with world renowned speakers at Blackburn (2016) and Chester (2018) cathedrals; a third series was delivered at Liverpool Cathedral in May 2019, and a fourth series is planned for May 2020.

My role is to consider all sciences (physical, clinical, social) in ecumenical and multi-faith environments.  So I will look to work with initiatives already developing in other Christian traditions, other faiths and secular organisations to discuss current challenges, such as climate change, medical ethics, health initiatives and information for cancer, dementia and mental health issues etc..

Mike Kirby2.jpg

My work will be part of the clear faith objectives of the cathedral as a place of encounter for everyone, through events and initiatives within the cathedral, but also beyond.  This will include services focusing on health issues and pastoral challenges (such as bereavement and loss); events engaging with science, its wonders and challenges; fostering further relationships with local and wider communities on science and healthcare education, and with academic and scientific institutions too; encouraging scientific and ethical engagement with schools and colleges, as I have done so previously in both Chester and Blackburn dioceses.

I will be encouraging Christians and Christian leaders to understand science and engage with it more, alongside other national projects such as the recently announced ECLAS (Engaging Christian Leaders in an Age of Science) project of Durham and York universities and the Church of England.  As a self-supporting minister (one whose paid employment is outside of the church), I will also look to encourage and highlight the tireless work of many others who already do this within the diocese and the wider national church.

Within all of this, I have always seen my vocation as being one within God’s service, for all people, with my work for cancer patients being right at the heart of it.

006

If you have any questions for Mike, you can send him an email at sigs@bir.org.uk

Mike is the co-author of the international student textbook on On-treatment Verification Imaging: a Study Guide for IGRT, through CRC press/Taylor and Francis with Kerrie-Anne Calder. They are both contributors to the updated UK national guidance on IGRT due out in 2020.

Mike, with the support of the SIG, has helped to organise a range of events for radiographers, physicists, dosimetrists, radiologists and oncologists. See the full programme here