Rx Strategist: Prescription Verification using LLM Agents System
Phuc Phan Van Dat Nguyen Minh An Dinh Ngoc Huy Phan Thanh
















Rx Strategist: Prescription Verification using LLM Agents System
    
Phuc Phan Van1, Dat Nguyen Minh1, An Dinh Ngoc1, Huy Phan Thanh1 
University Collaboration with Li Jinghong2, Dong Yicheng2 

1FPT University, Ho Chi Minh Campus, Vietnam
2Japan Advanced Institute of Science and Technology, Ishikawa Campus, Japan
{phucpvse170209, datnmse170570, andnse171386, huypt24}@fpt.edu.vn, and  {lijinghong-n, s2320035}@jaist.ac.jp
    September 9, 2024
========================================================================================================================================================================================================================================================================================================================================================================



Rx Strategist: Prescription Verification using LLM Agents System
    
Phuc Phan Van1, Dat Nguyen Minh1, An Dinh Ngoc1, Huy Phan Thanh1 
University Collaboration with Li Jinghong2, Dong Yicheng2 

1FPT University, Ho Chi Minh Campus, Vietnam
2Japan Advanced Institute of Science and Technology, Ishikawa Campus, Japan
{phucpvse170209, datnmse170570, andnse171386, huypt24}@fpt.edu.vn, and  {lijinghong-n, s2320035}@jaist.ac.jp
    September 9, 2024
========================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 

To protect patient safety, modern pharmaceutical complexity demands strict prescription verification. We offer a new approach - Rx Strategist - that makes use of knowledge graphs and different search strategies to enhance the power of Large Language Models (LLMs) inside an agentic framework. This multifaceted technique allows for a multi-stage LLM pipeline and reliable information retrieval from a custom-built active ingredient database. Different facets of prescription verification, such as indication, dose, and possible drug interactions, are covered in each stage of the pipeline. We alleviate the drawbacks of monolithic LLM techniques by spreading reasoning over these stages, improving correctness and reliability while reducing memory demands. Our findings demonstrate that Rx Strategist surpasses many current LLMs, achieving performance comparable to that of a highly experienced clinical pharmacist. In the complicated world of modern medications, this combination of LLMs with organized knowledge and sophisticated search methods presents a viable avenue for reducing prescription errors and enhancing patient outcomes.




Medical Systems, Large Language Model, Question Answering




§ INTRODUCTION






Verifying prescriptions is an essential stage in the healthcare process that guarantees patient safety and the best possible results from treatments. However, studies have shown that a significant proportion of prescribed dosages are erroneous. For instance, A work analysis of medication errors in Vietnamese hospitals <cit.> found that roughly 40% of doses prescribed to patients are incorrect in two urban public hospitals in Vietnam. Moreover, the availability of healthcare professionals, particularly in regions like Vietnam, is limited, exacerbating the issue. According to the Ministry of Health [https://moh.gov.vn/documents/174521/1760801/3.++BC-BYT-+T%E1%BB%95ng+k%E1%BA%BFt+ng%C3%A0nh.pdf/481b5482-2b3c-4487-bfd6-20dd2601cb04Ministry of Health (MOH) 2023 Report], Vietnam has just 12.5 doctors and 3.2 graduate pharmacists for every 10,000 people. This shortage of qualified personnel underscores the urgent need for advanced systems capable of automating and enhancing prescription verification without relying heavily on human resources.

Leveraging artificial intelligence (AI), particularly LLMs, as an assistant for healthcare providers offers a promising solution to mitigate prescription errors. AI-powered systems can rapidly analyze vast amounts of medical information, potentially identifying inconsistencies or potential issues with dosages, drug interactions, and contraindications.  However, current LLM systems face challenges in achieving reliable performance in this domain. Notably, the limited availability of real-world clinical data for AI model training raises concerns about their generalization to heterogeneous patient populations and varied clinical scenarios. Moreover, many LLMs rely on memorization rather than deep medical reasoning <cit.>, making them susceptible to hallucinations or incorrect answers when faced with unfamiliar or complex cases.

To address these challenges, we propose a novel LLM agent system designed specifically for prescription verification. Our system incorporates a sequence of specialized agents including 2 main tasks: indication verification and dose verification, each equipped with a unique combination of knowledge graphs, rule-based systems, and LLM components. This modular architecture enables a comprehensive analysis of each prescribed active ingredient, taking into account patient-specific information, the indicated condition, and established medical knowledge by combining the strengths of structured knowledge sources with the adaptability of LLMs. In addition, we introduce a specialized dataset focused on drug information and a novel methodology for knowledge retrieval. This dataset, combined with our retrieval approach, aims to enhance the system's robustness and overall performance. 

Chain-of-thought (CoT), introduced by <cit.>, marks a pivotal advancement in improving the reasoning abilities of text-generation models. CoT enables models to generate intermediate steps in their thought processes, mirroring human problem-solving techniques. Research by <cit.> further demonstrated that certain prompts, like "Let's think step-by-step," can naturally induce CoT reasoning in LLMs. These breakthroughs have laid the groundwork for ongoing research aimed at enhancing the reasoning capabilities of LLMs.

To address the inherent knowledge limitations of LLMs, Retrieval Augmented Generation (RAG) has emerged as a prominent approach. RAG integrates LLMs with information retrieval systems, enabling them to access and utilize relevant external knowledge. This is typically achieved by embedding both the query and candidate documents in a shared vector space and then identifying the documents with the highest similarity to the query. Recent advancements in RAG have focused on improving retrieval accuracy. For instance, HyDE <cit.> enhances retrieval by generating a hypothetical answer to the query and then embedding it alongside the documents, allowing for a more nuanced comparison. Alternatively, Take a Step Back <cit.> aims to identify the foundational knowledge documents most relevant to the query, potentially leading to more accurate and comprehensive responses.

Beyond traditional document retrieval, advanced RAG systems have increasingly turned to knowledge graphs (KGs) to improve retrieval accuracy and context understanding. KGs offer a structured representation of knowledge, capturing entities, relationships, and facts in a graph format. By integrating KG retrieval into RAG pipelines, researchers have been able to achieve several key advantages:

    
  * Structured Knowledge: KGs provide a structured representation of knowledge, enabling more precise retrieval and reasoning compared to unstructured text.
    
  * Semantic Understanding: KGs capture the semantic relationships between entities, allowing RAG systems to better understand the meaning and context of queries.
    
  * Multi-hop Reasoning: KGs can facilitate multi-hop reasoning, where the system navigates multiple relationships in the graph to answer complex questions.
    
  * Explainability: KGs can provide a transparent explanation of the reasoning process by highlighting the relevant entities and relationships used to generate a response.


Recent works such as <cit.> and <cit.> have demonstrated the effectiveness of incorporating KG retrieval into RAG. These systems leverage KG embeddings, graph traversal algorithms, and graph neural networks to identify relevant entities and subgraphs within the knowledge graph, providing the LLM with richer context and enabling more accurate and informative responses.

Besides optimizing in LLMs and RAG, many works integrate many LLMs together and also provide them tools for calling <cit.> to boost the performance of LLMs. These works demonstrated that a multi-agent system, where individual agents specialize in different reasoning tasks and communicate via function calls, can outperform single-agent models on complex question-answering tasks. This architecture allows for a more modular and adaptable approach, where agents can leverage each other's expertise and collaboratively generate responses. As a result, LLMs can benefit from improved accuracy, a wider range of capabilities, and better handling of complex tasks that require multiple perspectives.



In summary, our main contributions are as follows: (1) we introduce a novel system flow for prescription verification, leveraging a multi-agent LLM architecture combined with knowledge graphs and rule-based systems; (2) we provide a specialized dataset focused on drug information and a novel knowledge graph-based retrieval methodology that significantly enhances the performance of our system; and (3) our system demonstrates exceptional performance across various metrics from models to human labels, outperforming current LLMs and achieving parity with highly experienced clinical pharmacists.








§ DATASET


Current inference approaches for LLMs, such as CoT and ReAct <cit.>, primarily rely on the model's response capabilities. However, these methods can be prone to hallucination and illogical reasoning. To mitigate these issues, we propose leveraging relevant reference materials, including drug and indication information, to guide LLM responses.


Instead of assessing specific drugs independently, we delve deeper into the active ingredients, which are specific compounds that make up a medication and are responsible for the therapeutic effects of it. For instance, Losartan is primarily used to treat hypertension and to protect the kidneys from damage due to diabetes, as well as helps relax blood vessels, making it easier for the heart to pump blood and reducing the risk of strokes and heart attacks [https://www.drugs.com/losartan.htmlLosartan information].



 §.§ Data Collection
 

























  §.§.§ Drug Information Sources

For this task, we collected highly accurate drug information from reliable sources like Drugs.com and Long Chau Pharmacy.

    
  * Drugs.com: Information on over 1700 active ingredients in AHFS DI Monographs format, including medication indication, administration, dosage, and adverse effects (if available).
    
  * Long Chau Pharmacy: Vietnamese-language information on medicinal properties specific to Vietnam.
Data from both sources was stored and retrieved as HTML documents, with each document corresponding to a specific active ingredient. The data was then chunked according to headers and saved in Markdown format for human readability. For evaluation and querying purposes, the Markdown documents were further processed into a structured JSON format.










  §.§.§ Standardizing Indication Terminology

To address inconsistencies in medical indication terminology, By using AI models like GPT 3.5 to generate ICD-10 codes based on various indication terms, we create a unified language for identifying and classifying diseases. This standardization allows healthcare professionals to quickly and easily identify the correct ICD-10 code, streamlining tasks such as medical billing, insurance claims processing, and research data analysis. In addition, accurate disease classification facilitated by standardized terminology can lead to improved diagnosis, treatment, and ultimately, patient outcomes.



  §.§.§ Drug Interaction Data

We collected interaction data for 27 common active components, with detailed name in Appendix <ref>, from Drug.com interaction checker. The data include interaction level and detailed descriptions of their interactions with other components. By integrating this data, we aim to enhance the model's ability to identify and evaluate potential adverse effects, ensuring safer and more effective prescription recommendations.




 §.§ Expert-Approved Labels
 
















To ensure safe and accurate prescription verification, we leveraged the expertise of clinical pharmacists with varying levels of experience to establish a robust and reliable evaluation process. To assess the model's performance at different levels of human expertise, we enlisted three clinical pharmacists: a junior pharmacist with one year of experience, an intermediate pharmacist with three years of experience, and a senior pharmacist with five years of experience. Each pharmacist independently evaluated prescriptions, verifying the appropriateness of both the indication and dosage for each active ingredient. If an indication was deemed incorrect, the corresponding dosage was not evaluated. Our diligent human labeling process not only provided valuable insights but also optimized the evaluation of our automated verification model. 

A council of three highly experienced hospital pharmacists (with over five years of experience) provided the definitive assessment used as the gold standard for accuracy to which individual pharmacists' results were compared. This comprehensive evaluation strategy ensures that our system not only aligns with but surpasses industry standards, fostering trust in its ability to enhance patient safety and optimize medication management. 







 §.§ Data Quality Control
 















For practical application and seamless integration into a database, the collected data underwent a rigorous preparation and quality control process. The initial Markdown format was converted into a structured JSON format for enhanced usability and compatibility.  This transformation involved several key steps:

    
  * Removal of extraneous characters: Non-essential symbols, including tabs (\t) and daggers (†), were eliminated. Newline characters (\n) were retained to preserve content separation.
    
  * Hyphen standardization: Long and short hyphens were unified to ensure consistency.
    
  * Dosage unit conversion: Regular expressions were employed to convert dosage amounts expressed in micrograms (mcg) to the standard unit of milligrams (mg).
    
  * Restructuring of JSON hierarchy: Redundant layers within the JSON structure were removed to streamline data access and manipulation.







 §.§ Data Statistics
 



The dataset includes 1780 active ingredients, each containing information about compatible age groups alongside usage, as shown in Figure <ref>.


  §.§.§ Age groups

Close assession and filtering of dataset reveals that 1694 (95%) of all available active ingredient can be used by adults, while only 923 (52%) of them are prescribed towards pediatric patients. This imbalance of data can be understandable as the number of illnesses grows positively correlated with a person's age.


  §.§.§ Total information in each active ingredient

In order to measure how much information is described in each medicine, we calculate the number of words available in the "description" portion of the data. The result shows that the number of words range from 0 up to 7697, with the majority of active ingredient descriptions fall under 1000 words. This indicates a large variance across our collection of active ingredient.


  §.§.§ Drug versatility

The visualization also includes drug versatility measurement, which equals the total number of diseases one drug can cure, according to our dataset. On average, each drug can be indicated to cure 3 diseases, and few of them can reach up 40. 

















§ METHODOLOGY


Overview. The Rx Strategist, as illustrated in Figure <ref>, is composed of three agents, an information extractor, and a checker. These components interact based on fit status, a key concept in the system, with the fit of indication (FI) and fit of dosage (FD) playing a crucial role in identifying the active ingredients and connecting the different modules. Specifically, a prescription is first processed by using optical character recognition (OCR) to extract relevant information, which is then converted into features by an LLM. ICD Finder is an LLM component identifies ICD-10 codes from the extracted active ingredients. ICD Matcher compares the identified ICD-10 codes with those in the patient's diagnosis, filtering the active ingredients to those that are suitable for the patient's condition as FI. Dosage Retriever determines the appropriate dosage of the active ingredient based on both FI and patient information. Finally, Checker utilizes both FI and FD to assess the overall validity of the prescription, providing explanations for its determination.




 §.§ Indication: Finding and Mapping










To initiate the prescription verification process and provide the necessary input for the ICD Finder, we first extract relevant information from the prescription image. The image of a prescription is fed into an OCR model, returning a full-text format of the whole prescription. Subsequently, this body of text goes through a reformatting phase using GPT-4o-mini, turning it into a dictionary-like format (with information such as age group, indication, and dosage) for easy extraction in the latter phase.

ICD Finder. The ICD Finder receives a list of active ingredients in dictionary format. To address the challenge of inconsistent name representation, we employ a fuzzy matching algorithm to identify potential matches between the received active ingredients and the database entries. This fuzzy matching approach enhances the system's ability to recognize active ingredients with slight variations in naming conventions, thereby improving the accuracy of subsequent steps. Once potential matches are identified, the ICD Finder retrieves the corresponding usages for each active ingredient, indicating the diseases each active ingredient is designed to treat. Subsequently, the ICD Finder maps these diseases to their respective ICD-10 codes, establishing a set of ICD-10 codes associated with the prescription's intended therapeutic purposes.

In instances where information about a specific active ingredient is unavailable within our database, we leverage the capabilities of an LLM to generate potential ICD-10 codes. The LLM is trained on a vast corpus of medical literature and clinical data, enabling it to infer potential ICD-10 codes based on the active ingredient's name, chemical structure, and known therapeutic uses. However, it is important to note that LLM-generated ICD-10 codes should be treated with caution and may require additional validation before being used for clinical decision-making.

ICD Matcher. Leveraging the ICD-10 codes obtained by the ICD Finder, the ICD Matcher compares them against the ICD-10 codes derived from the patient's diagnosed conditions. The comparison is conducted at the category level of the ICD-10 code, which consists of an alphabetical letter followed by two numbers (e.g., A01, B15, C23). An active ingredient is labeled as "APPROPRIATE" if all ICD-10 categories associated with its usages are present in the patient's diagnosis codes, signifying that it aligns with the patient's medical needs. Conversely, if any ICD-10 category linked to an active ingredient's usage is absent from the patient's diagnosis codes, indicating a potential mismatch between the medication and the patient's condition, the active ingredient is classified as "INAPPROPRIATE".




 §.§ Dosage: Retriever with Knowledge Graph


Due to the structured nature of our dataset and the possibility that LLMs might overlook subtle textual details <cit.>, we have developed a specialized text-to-graph processing approach. This method focuses on task-specific information extraction, improving the robustness of our model while minimizing the need for extensive fine-tuning.

We extracted dosage information for each disease within specific age groups (e.g., pediatric, adult) and administration routes (e.g., oral, intravenous) from our dataset. Given the absence of detailed patient histories in our prescription data, we established the initial dosage specified in the drug information as the recommended baseline dose. Any prescribed dosage deviating from this baseline was flagged for further scrutiny, as it might necessitate individualized adjustments based on the patient's specific circumstances and medical history. To structure this information, we constructed a knowledge graph with nodes representing drugs, diseases, and dosages, and edges denoting the relationships between them (see Appendix <ref> for details). The process of generating this knowledge graph is outlined in Algorithm <ref>. Specifically, the algorithm iterates through each active ingredient in the dataset, then for each age group associated with that ingredient, and finally for each disease that the ingredient is indicated for within that age group. For each disease, the language model is used to generate dosage nodes and relationships, which are then added to the knowledge graph. This structured representation facilitates efficient retrieval and reasoning about dosage information, enhancing the accuracy and effectiveness of our prescription verification system.








Dosage Retriever. a knowledge graph-based system, efficiently retrieves appropriate dosages for verified active ingredients. It takes as input the active ingredients validated in the indication stage, along with patient-specific details like age group, age-specific factors (e.g., weight, kidney function), and the diagnosed condition the drug aims to treat. The system then navigates the knowledge graph, which is structured around relationships between drugs, diseases, and dosages. The retrieval process involves comparing the patient's information with the knowledge graph's nodes and edges. For instance, if the patient is a child diagnosed with a specific condition, the Dosage Retriever will traverse the graph to find the active ingredient node, follow the edge to the relevant disease node, and then identify the dosage node associated with the child age group. To enhance accuracy, a language model is integrated to standardize keywords and address inconsistencies in drug information representation. This ensures precise matching between the patient's data and the knowledge graph's information. If no exact match is found, the language model can suggest the closest available dosage information.

Algorithm <ref> details the step-by-step process of dosage retrieval. It first identifies relevant diseases for the active ingredient and patient age group within the knowledge graph. If the diagnosed disease isn't directly linked, the language model assists in finding the closest match. The patient's age is then categorized into a specific age range, and potential dosages are retrieved based on the active ingredient, disease, and age range. If multiple dosage options exist, the system selects the most appropriate one based on the patient's specific age and any additional guidance from the language model. If no dosage information is available, the system indicates this lack of information.










































§ EXPERIMENTAL SETTINGS



 §.§ Benchmarks

To evaluate the performance of our system, we curated a dataset of 20 real-world prescriptions sourced from Vietnam hospitals, the age group of patient is majoring as adults. To ensure patient privacy, all personally identifiable information, including names, addresses, and hospital details, was meticulously removed from the dataset.



 §.§ Baselines and Experimental settings


Assessing our system’s capabilities reliably and comprehensively involved comparing its performance to a diverse set of benchmarks, including state-of-the-art language models (LLMs) and human experts. This multifaceted approach allowed for a robust evaluation across various metrics and perspectives. For the LLM baseline, we utilized both open-source (Qwen2 72B by Alibaba group <cit.>, LLama3.1 family models from 8 parameters to 405 parameters by Meta <cit.>) and closed-source models (GPT4o-mini by OpenAI[<https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/>] and Claude 3.5 Sonnet by Anthropic[<https://www.anthropic.com/news/claude-3-5-sonnet>]). To evaluate human performance, we collected prescription evaluations from clinical pharmacists with 1 to 5 years of experience, establishing a real-world benchmark.

All LLM evaluations employed a CoT prompting method, with their specific prompts detailed in Appendix <ref>. Open-source models were initially configured with a temperature of 0, top-p of 0.7, and top-k of 50 for all models except LLama 3.1 8B. Additionally, We observed a tendency for LLama 3.1 8B to omit answers and duplicate words for certain prescription types. To address this, we adjusted the temperature for LLama 3.1 8B to 0.2 or 0.3 for prescription inference, and a value of 0.5 for interaction query summarization, encouraging a more exploratory approach and potentially leading to more comprehensive responses. Closed-source models were utilized with default settings provided by their respective developers.

Our system utilizes GPT4o-mini as the underlying language model due to its availability, ease of integration, and strong performance in preliminary tests.







 §.§ Evaluation Metrics

To assess the performance of our system, we prepare a variety of metrics including accuracy, precision, recall, F-0.5 score as our evaluation metrics. For each prescription, we compare the set of active ingredients predicted by our system against the corresponding gold standard set (i.e., the ground truth active ingredients).







Basic metrics such as Accuracy, while reliable for most tasks, are not the best indicator of good performance for this task. More specifically, cases of inappropriate elements being flagged as appropriate are considered harmful to patients, as they create a false belief that their description is accurate when it is not. On the other hand, the opposite case of appropriate elements being flagged as inappropriate is less serious, as it only requires a close assession by the professionals. Therefore, we introduce more metrics to put emphasis on the harmful case.

The list of metrics we will use to assess performance is the following:

    
  * Accuracy: proportion of correct prediction over all samples.
    
  * Precision: proportion of predicted appropriate elements that are correct.
    
    
    
    
  * Recall:  proportion of actual appropriate elements that are correctly predicted.
    
    
    
    
  * F-0.5 Score: variant of F-β Score with β=0.5, measuring weighted harmonic mean of precision and recall. Instead of the traditional F1-Score, which balances precision and recall, this variant weights more on precision, with the purpose of minimizing precision - inappropriate elements classified as appropriate.
    
    
    









§ RESULTS



Table <ref> presents a comprehensive comparison of our system's performance against human labels, open-source models, and closed models. The results unequivocally demonstrate that our system surpasses nearly all current LLMs, achieving a level of knowledge comparable to that of a clinical pharmacist with 5 years of experience in which you can see more in Figure <ref>.  In particular, our system outperforms GPT4o Mini by 5.56% and Claude3.5 Sonnet by 3.71% in terms of accuracy on closed models. On human labels, Rx Strategist outperforms clinical pharmacist with one year of experience by 4.63% and by 3.71% for clinical pharmacist with three years of experience, highlighting its superior capability in this domain.

Interestingly, we observed a positive correlation between the precision-recall ratio and the F-0.5 score in prescription verification tasks (Figure <ref>). This suggests that models prioritizing precision (minimizing false positives) while maintaining a reasonable recall (minimizing false negatives) tend to achieve higher overall performance, as indicated by the F-0.5 score. This finding highlights the importance of balancing precision and recall in this domain, where both accurate identification of valid prescriptions and avoidance of incorrect rejections are crucial. Our system, Rx Strategist, demonstrates this balance effectively, positioning it as a promising tool for enhancing prescription verification accuracy and ultimately improving patient safety.





Another noteworthy performance criterion is runtime, which is an important indication of how long our system can return its output. Table <ref> clearly demonstrates the difference between the time taken by Rx-Strategist compared to other LLms, with additional information such as average time per token and total generated tokens for measuring conciseness.

calculated for LLM only






§ INTERACTION CHECKING


Given the complex nature of interaction between medical elements, we propose a new way of representing the relationship of medical properties by utilizing a Knowledge Graph. 
The triplets, which is used for graph construction, is extracted using the LLama3-8B model with prompting that targets domain-specific relationships, as detailed in the Appendix <ref>. Subsequently, the corresponding triplet embeddings are generated using the general text embedding model 
gte-Qwen2-1.5B-instruct <cit.>. 



To validate the approach's performance against other experimental results and determine its effectiveness for our specific application, we have Knowledge Graph's impact on refining evaluation outcomes. The graph is combined with two models LLama3-8B <cit.> and LLama3.1-8B <cit.> respectively. Information on active components in prescriptions is retrieved from the graph using the cosine similarity score between the input query embedding and the precomputed triplet embeddings within the graph. The retrieved data is subsequently summarized using the same inference model to condense the information. We anticipate that this approach will enable the model to interpret the graph from its perspective. The summarized interactions are then utilized to aid the LLM in its prescription evaluation. The result shows that both LLama3-8B and LLama3.1-8B have improved their accuracy substantially, achieving an increase of 8.33%.
































§ CONCLUSION AND FUTURE WORK


In conclusion, we present a novel, efficient approach to prescription verification, effectively combining a knowledge base with a well-instructed reasoning process. Our system's performance surpasses not only some senior clinical pharmacists but also state-of-the-art LLMs, demonstrating its potential for real-world implementation, particularly in resource-constrained hospital settings.

Limitations and Future Works: Several avenues for future enhancement exist. The current reliance on Vietnamese-language data necessitates further investigation into multilingual capabilities to ensure broader applicability. Additionally, refining the ICD-10 coding process, potentially through the development of a dedicated model, could mitigate the risk of hallucination and further improve accuracy. Expanding the knowledge base to incorporate diverse data sources, such as electronic health records and clinical guidelines, could enhance the system's understanding of complex medical scenarios, ultimately leading to more robust and reliable prescription verification.


ieeetr  







 §.§ LLM Prompt for Evaluate




  §.§.§ Base evaluation prompt

 

You are a meticulous clinical pharmacist specializing in medication safety and appropriateness. Given a patient's
profile and prescription, your task is to thoroughly evaluate the prescription's suitability.
Pay close attention to the patient's age, medical conditions (especially kidney failure, liver disease, and
pregnancy), and any relevant allergies.
Follow these steps for each medication in the prescription:
1. Assess Patient Profile: Carefully review the patient's information to identify any potential risk factors
or contraindications.
2. Verify Indication: Determine if the medication or combination of medication is APPROPRIATE, INAPPROPRIATE, or
UNDERPRESCRIBED for the patient's diagnosed condition(s).
3. Verify Dosage and Administration (If Appropriate): If the medication is appropriate, confirm that the prescribed
dosage and administration instructions are safe and effective for the patient.
4. Conclusion: For each medication, provide a final assessment:
* If APPROPRIATE, state "APPROPRIATE".
* If INAPPROPRIATE, specify which aspect (e.g., dosage, active ingredient, interaction) is problematic and
provide a detailed explanation.
Prescription:




  §.§.§ Evaluation prompt with Interaction graph



You are a meticulous clinical pharmacist specializing in medication safety and appropriateness.
Given the reference materials bellow.
—————————————-
Indication query summarization
—————————————-
Your task is to thoroughly evaluate the prescription's suitability in english.
Pay close attention to the patient's age, medical conditions (especially kidney failure, liver disease, and
pregnancy), and any relevant allergies.
Follow these steps for each medication in the prescription:
1. Assess Patient Profile: Carefully review the patient's information to identify any potential risk factors or
contraindications.
2. Verify Indication: Determine if the medication or combination of medication is APPROPRIATE, INAPPROPRIATE,
or UNDERPRESCRIBED for the patient's diagnosed condition(s).
3. Verify Dosage and Administration (If Appropriate): If the medication is appropriate, confirm that the prescribed
dosage and administration instructions are safe and effective for the patient.
4. Conclusion: For each medication, provide a final assessment:
* If APPROPRIATE, state "APPROPRIATE".
* If INAPPROPRIATE, specify which aspect (e.g., dosage, active ingredient, interaction) is problematic and
provide a detailed explanation.
Prescription:





 §.§ Components table







 §.§ Data Representation



  §.§.§ Raw JSON data Sample


[breaklines=True]
"losartan": 
    "dosage": 
        "Pediatric Patients": 
            "Hypertension": 
                "Oral": "Children >6 years of age.. ."
            
        ,
        "Adults": 
            "Hypertension": 
                "Losartan Therapy": "Oralrecommends initial dosage of 50 mg..."
            ,
            "Prevention of Cardiovascular Morbidity and Mortality": 
                "Oral": "Initially, 50 mg once daily..."
            ,
            "Diabetic Nephropathy": 
                "Oral": "Initially, 50 mg once daily..."
            ,
            "Heart Failure [off-label]": 
                "Oral": "Initially, 25-50 mg once daily ..."
            
        
    
,
"esomeprazole": 
    "dosage": 
        "Pediatric Patients": 
            "GERD": 
                "GERD Without Erosive Esophagitis": "Oral1-11 years of age..."
            
        ,
        "Adults": 
            "GERD": 
                "GERD Without Erosive Esophagitis": "Oral20 mg once daily ..."
            ,
            "Duodenal Ulcer": 
                "Helicobacter pylori Infection and Duodenal Ulcer": "Oraltherapy:..."
            ,
            "NSAIA-associated Ulcers": 
                "Prevention of Gastric Ulcers": "Oral20 or 40 mg once daily;..."
            
        
    
,
...






 §.§ Prescription Sample

An example of a prescription named 24th is demonstrated in Figure <ref>.





 §.§ Dosage Knowledge Graph Structure




  §.§.§ Node Examples


[breaklines=True]
[
    
        "id": 1,
        "name": "rosuvastatin",
        "type": "Drug"
    ,
    
        "id": 2,
        "name": "heterozygous familial hypercholesterolemia",
        "type": "Disease"
    ,
    
        "id": 3,
        "name": "5-10 mg once daily",
        "type": "Dosage"
    ,
    ...
]




  §.§.§ Relationship Examples


[breaklines=True]
[
    
        "start_id": 1,
        "end_id": 2,
        "type": "TREATS",
        "age_group": "pediatric"
    ,
    
        "start_id": 2,
        "end_id": 3,
        "type": "INITIAL_DOSAGE",
        "age_specific": "children 8 to <10 years of age",
        "administration": "oral"
    ,
    
        "start_id": 2,
        "end_id": 4,
        "type": "INITIAL_DOSAGE",
        "age_specific": "children and adolescents 10-17 years of age",
        "administration": "oral"
    ,
    
        "start_id": 1,
        "end_id": 5,
        "type": "TREATS",
        "age_group": "adults"
    ,
    
        "start_id": 5,
        "end_id": 6,
        "type": "INITIAL_DOSAGE",
        "administration": "oral"
    ,
    
        "start_id": 5,
        "end_id": 7,
        "type": "SPECIFIC_DOSAGE",
        "administration": "oral",
        "indication": "patients who have not achieved adequate response with the 20-mg daily dosage"
    ,
    
        "start_id": 1,
        "end_id": 8,
        "type": "TREATS",
        "age_group": "adults"
    ,
    ...
]






 §.§ Output Examples



  §.§.§ LLama3.1 405B

The output of the LLama 3.1 405B model on the 24th prescription (one of the prescription in the evaluation dataset). For more information about input, see at Appendix <ref>

I'll evaluate each medication in the prescription based on the patient's profile.
Patient Profile:
- Age: 45 (born in 1978)
- Medical Conditions:
- Chronic hepatitis B without hepatitis D coinfection
- Non-insulin-dependent diabetes mellitus (without complications)
- Mixed hyperlipidemia
- Hypertension
- No allergies mentioned
Medications:
1. Tenofovir (TDF) - Tefostad T300 300mg
- Assess Patient Profile: Patient has chronic hepatitis B, which Tenofovir is indicated for.
- Verify Indication: APPROPRIATE for chronic hepatitis B.
- Verify Dosage and Administration: The prescribed dose (300mg once daily) is standard for treating chronic
hepatitis B.
- Conclusion: APPROPRIATE
2. Atorvastatin - Lipotatin 20mg
- Assess Patient Profile: Patient has mixed hyperlipidemia.
   
- Verify Indication: APPROPRIATE for hyperlipidemia.
   
- Verify Dosage and Administration: The dose (20mg once daily) is within the recommended starting dose range
for adults.
   
- Conclusion: APPROPRIATE

3. Amlodipin + losartan - Troysar AM 5mg + 50mg
   
- Assess Patient Profile: Patient has hypertension.
   
- Verify Indication: APPROPRIATE for hypertension management.
   
- Verify Dosage and Administration: The combination and doses (5mg of Amlodipine and 50mg of Losartan once
daily) are within recommended ranges for hypertension management.
   
- Conclusion: APPROPRIATE

4. Metformin hydroclorid - Meglucon 1000 1000mg
   
- Assess Patient Profile: Patient has non-insulin-dependent diabetes mellitus.
   
- Verify Indication: APPROPRIATE for type 2 diabetes.
   
- Verify Dosage and Administration: The dose (1000mg twice daily) is a common starting dose for adults.
   
- Conclusion: APPROPRIATE

5. Linagliptin - TRIDJANTAB 5mg
   
- Assess Patient Profile: Patient has non-insulin-dependent diabetes mellitus.
   
- Verify Indication: APPROPRIATE for type 2 diabetes.
   
- Verify Dosage and Administration: The dose (5mg once daily) is the recommended dose for adults.
   
- Conclusion: APPROPRIATE

Overall Assessment: All medications in the prescription are APPROPRIATE for the patient's conditions, with no
contraindications based on the provided information. However, it's essential to monitor kidney function, especially
with the use of Tenofovir and Metformin, and to assess the efficacy and potential side effects of all medications
regularly.








































































































































































































































































