Image extraction followed up with text extraction from the image

Automating Insurance Card Image Extraction

The Client

Healthcare RPA company

One of the most trusted names in the development of healthcare IT systems, this US-based healthcare RPA company wanted us to extract pertinent information from insurance cards.

Key challenges faced by the customer

Image identification from the document 0%
Information extraction from the image 0%

Executive Summary

Industry Overview


The global healthcare IT market size was valued at USD 135.6 billion in 2021 and is projected to exhibit a CAGR of 29.3% in the forecast period. A growing trend of digitalization in healthcare, rising demand for preventive care solutions, and the emergence of various entrepreneurship ventures are propelling the market’s growth. With the support of an increasingly sophisticated IT infrastructure, key participants are focusing on innovative product development strategies and making constant technological advancements. To enhance health & medical processes, market players are integrating big data, the Internet of Things (IoT), artificial intelligence (AI), and machine learning (ML) algorithms into their existing solutions.
Business Challenge

Customer Experience

In the IT healthcare industry, one of the key operations is to extract relevant information from different types of documents. The customer was facing challenges with the traditional method to extract insurance cards from images and retrieve only the relevant information from those extracted insurance cards. These challenges were mainly due to the following reasons: Primarily, the conventional method was rule-based and could not work with newer or different formats (more than 8000) of insurance cards. Moreover, with very skewed labelled data the rule based approach functioned only approximately 50% of the time. Secondly, the process was very slow as OCR for high-resolution images consumes a lot of time. The client wanted us to automate this extraction process with low latency and high accuracy along with the task to identify the type of insurance.

The Akaike Edge

Inbuilt libraries, DL models with transfer learning capabilities

Key solutions built for the customer

Automated object detection for insurance card image 0%
Faster DL algorithm for information extraction 0%
Automated process for classifying the type of insurance 0%


Inbuilt libraries, DL models with transfer learning capabilities
Step 1.

Adjusted working process to adhere to HIPAA compliance

Before beginning to solve the problem, Health Insurance Portability and Accountability Act (HIPPA) compliance was to be respected. We then had to work remotely using virtual machines as per the client's environment. To adjust to this working process, a few labeling tools were built to maintain HIPAA compliance

Step 2.

Utilized object detection-based models to extract the insurance card from the image and LayoutLMV2 for information extraction.

We iterated with several segmentation and object detection-based algorithms to crop insurance cards from the given images, and then extract pertinent information from the insurance cards. Additionally, this insurance card extraction from the provided image was completed with a very low latency rate of 20 ms per image. Following that, LayoutLMV2, a transformer-based model for labelling the information such as Ids, Names, etc., received the extracted card image and text as input. Eventually, we improved the process' speed, dynamic nature, and accuracy by feeding the model additional relevant data. Finally, the classification model was used to this extracted insurance card image to determine the payer type.

Step 3.

Model deployment for object detection and information retrieval

The customized model was deployed with Rest API using FastAPI in the client’s AZURE environment using VMSS that automatically scales up and down the virtual machines as per the requirement. The whole process of insurance card extraction and information retrieval is now automatic, more accurate, and faster.

Research shows that 87% of Vision AI
projects do not yield expected results

either owing to training data insufficiencies stalling the project, or being too slow to deploy. Our AI experts can help you accelerate in data sparse environments.

We use cookies to give you the best experience.

Get in Touch