How to Implement AI Healthcare Applications: Technical Architecture, Model Selection, and Risk Control Essentials

By 2026, the typical approach for building AI medical/health applications is to first define the business scenario (e.g., tongue and facial diagnosis, report interpretation), then build the capability layer on top of multimodal large models (such as GPT-5.6, Tongyi Qwen 3.7, DeepSeek V4), connect to medical knowledge bases via RAG, and finally deliver to WeChat Mini Programs, APPs, or H5 carriers, with data desensitization and content review added at the periphery. Key modules include visual recognition, structured consultation, report generation, and risk alerts—all are indispensable.

1. Standard Technical Architecture for AI Medical/Health Applications

By 2026, the mainstream architecture of medical/health AI applications has stabilized, with most adopting a four-layer decomposition: "Capability Layer → Business Layer → Carrier Layer → Data and Risk Control." This division decouples model capabilities from business logic, making it easier to replace models or add new scenarios later.

• Capability Layer: Multimodal large models (vision, language) + Medical knowledge base (RAG) + Tool invocation (e.g., OCR, structured output). Note that vision models need to support medical image feature recognition, and language models need to support long context and multi-turn dialogue.
• Business Layer: Consultation flow, report generation, constitution analysis, health suggestions, anomaly alerts. Domain rules need to be accumulated here, such as the Eight Principles (Ba Gang) system in TCM diagnosis.
• Carrier Layer: WeChat Mini Programs, Official Account H5, APPs, websites, or integration into healthcare systems. It is recommended to start with the WeChat ecosystem to lower user adoption barriers.
• Data and Risk Control: Data desensitization, permission management, hallucination filtering, compliance review (medical qualifications and advertising laws). This step determines whether the product can operate in the long run.

Why this division? Because each layer has its own responsibility: the capability layer handles "understanding and seeing," the business layer handles "how to respond," the carrier layer addresses "where to use," and data risk control provides the bottom line to "avoid problems." Teams can develop and test modules in parallel.

2. How to Implement Key Modules: Tongue and Facial Diagnosis, Report Interpretation, and Q&A

Taking composite vision+language scenarios such as "tongue/facial diagnosis" as an example, the core process is image acquisition, preprocessing (contour/color correction), feature extraction (tongue color/coating texture), and combining with user questionnaires to generate constitutional judgments. By 2026, the common approach is to first use a vision model (e.g., Gemini 3.6 Flash) for basic feature recognition, then use a language model (e.g., Tongyi Qwen 3.7) to integrate textual descriptions, and finally apply TCM syndrome differentiation rules at the business layer.

Actionable suggestions: Ensure uniform lighting in the acquisition environment to avoid color distortion; model outputs must be validated with rules, e.g., the tongue color cannot be both pale white and dark purple simultaneously; reports must include a disclaimer stating "For reference only, not a diagnosis." The acceptance criterion is that the recognition results achieve an agreement rate of over 80% with professional physician annotations on the test set (the specific threshold is confirmed by the project team and professional medical consultants).

For report interpretation features, you can do this: have users upload lab reports, use OCR to extract indicators, compare them against reference intervals to mark abnormalities, and finally use a language model to generate plain-language explanations with a reminder to "consult a doctor." It is recommended to introduce a "structured output" mechanism, where the model returns indicator names, values, and statuses in JSON format, to avoid parsing errors caused by free-form text.

3. Model Selection and Invocation: API, Private Deployment, or Hybrid?

By 2026, there are many model APIs available. API calls are low-cost and quick to launch, but medical data is sensitive, and private deployment is more compliant. Based on project specifics, we compare four dimensions: cost, timeline, data privacy, and update frequency.

• API Calls: Billed per token, with large models costing roughly RMB 0.5-2 per thousand tokens (depending on the model and plan); launch timeline is 1-3 weeks; data must be desensitized before transmission; model updates are immediate.
• Private Deployment: Requires GPU resources (e.g., a single A100 or dual 4090 cards), with first-year costs around RMB 200,000-600,000; launch timeline is 1-2 months; data stays within the domain; model versions require self-maintenance.
• Hybrid Approach: Business requests go through APIs, while sensitive data is processed by private models, balancing cost, compliance, and effectiveness. For example, tongue diagnosis images can be desensitized and sent to cloud-based vision models, while consultation dialogues use a private language model.

Conclusion: For health consultation (non-diagnostic) use cases, APIs can be used for quick validation; for scenarios involving patient privacy or internal systems of tertiary hospitals, direct private deployment is recommended. If the team lacks operational experience, consider using a hybrid architecture to first establish the business process.

4. Data and Risk Control Issues to Address Before Launch

The main risks of medical/health AI are "hallucinations" and compliance issues. In 2026 project delivery practices, multiple lines of defense must be established: the retrieval sources in the knowledge base must be authoritative textbooks/guidelines (e.g., TCM Diagnostics); language model outputs must go through rule-based filtering; high-risk words (such as "cure" or "eradicate") must trigger manual review or refusal to answer.

The risk control checklist is as follows:

• Data Desensitization: Remove fields such as name, ID number, and phone number; facial features in images can be pixelated or cropped, retaining only the tongue/facial skin area.
• Output Review: Perform a filter based on sensitive word lists and regex rules; refuse to answer low-confidence results (e.g., directly reply "Please seek offline medical consultation").
• Qualification Boundary: Do not provide any diagnostic conclusions, only output "tendency" and "suggestions"; remind users that the product is an auxiliary tool and cannot replace a doctor.
• Log Retention: Keep conversation records and model versions for traceability and review.

Acceptance criteria: the hallucination rate, manually evaluated on a test set, should be controlled below 1%, and all answers must include a disclaimer; the risk control interception rate should cover all high-risk words (i.e., either refuse or trigger manual review).

FAQ

When selecting a model, should we use a general-purpose large model or a medical-specific model?

General-purpose models (e.g., GPT-5.6, DeepSeek V4) are more flexible in common-sense Q&A, while medical-specific models (e.g., medical versions fine-tuned from Qwen) are more accurate in terminology and diagnostic suggestions, but you need to evaluate effectiveness and cost yourself.

What is the approximate cost of medical AI applications?

API call first-year costs range from tens of thousands to hundreds of thousands of RMB, while private deployment starts at around RMB 200,000-600,000 in the first year, depending on concurrency, model specifications, and whether custom fine-tuning is needed.

How can AI "hallucinations" be reduced in medical scenarios?

Use RAG to force retrieval from authoritative knowledge sources, apply rule-based validation to model outputs, and add a fallback logic of "refuse to answer when uncertain," while keeping a manual review entry point.

How to decide between private deployment and API calls?

For scenarios with high data sensitivity, private deployment is recommended; for projects with short development cycles, where data can be desensitized, and cost-sensitive, you can validate with APIs first, then gradually transition to a hybrid architecture.

What are the core metrics for launch acceptance of medical AI applications?

Focus on answer accuracy, recall rate, hallucination rate, response time (e.g., P95 less than 3 seconds), risk control interception rate, and whether medical content review is passed.

Applicable Scenarios and Boundaries

This solution is suitable for non-diagnostic scenarios such as health management, TCM constitution analysis, physical examination report interpretation, and intelligent triage, improving efficiency and assisting doctors. It is not suitable for directly providing treatment plans, prescribing, replacing doctor diagnosis, nor for teams without medical qualifications to promote "curing all diseases." Any part involving prescriptions and diagnosis must involve a licensed physician.

Boundary statement: The value of medical AI applications lies in assistance and efficiency improvement, not replacing professional medical behavior; if the product definition crosses the "diagnosis and treatment" red line, no matter how advanced the technology, it cannot be compliantly launched.

Action Guide: It is recommended to run a small-scale pilot based on the above four-layer architecture, iterating models and processes with real user feedback. For startup teams, consider using APIs plus standard SaaS components to get an MVP running; if the goal is to enter the hospital system, then consider private deployment or collaborate with delivery teams like Xiyue Company to design a compliant architecture. Remember: first establish one business line, then expand horizontally—don't try to roll out all features at the beginning.