git_admin/obsidian-vault

Fork 0

Files

T

Alexey Martemyanov c5f2d6bb14 [2026-05-31] personal-os notes, psychologist-app planning

2026-05-31 04:01:42 +06:00

32 KiB

Raw Blame History

title, aliases, created, updated, tags, related

title

aliases

created

updated

Crisis Protocol — AI Reflection App

Researched: 2026-05-16
Status: Draft v1.0 — implementation-ready spec
Audience: Engineering + product
Priority: HARD BLOCKER — required for App Store approval

Apple App Store Requirements (§1.4.x + Health)
How Approved Apps Do It
Technical Implementation Options
Keyword Lists — English & Russian
Crisis Resources by Country
Legal Liability & Disclaimers
Implementation Spec — What to Build

1. Apple App Store Requirements

What the Guidelines Actually Say

Apple's App Store Review Guidelines (last updated February 6, 2026) address mental health apps under Section 1.4 (Physical Harm) and through the broader Health & Fitness app category policies. The directly relevant provisions:

1.4.1 — Apps designed to provide medical / health services that could result in physical harm if incorrect must be able to demonstrate accuracy. Mental health apps receive greater scrutiny here.

1.4.2 — Drug dosage calculators must come from approved sources (manufacturer, hospital, university, pharmacy, FDA-approved entity). This is the section the task originally references — but note: the crisis protocol requirement for mental health apps is typically enforced through a combination of §1.4 and the Health & Fitness guidelines, not solely §1.4.2 (which technically addresses dosage calculators). The spirit of §1.4 (avoid harm) is why the crisis protocol is required.

The actual mental health / crisis protocol mandate comes from Apple's applied review practice under §1.4 and the Health & Fitness category standards. Apple review notes consistently state:

"Apps designed to provide mental health support must be able to refer users to relevant emergency services or provide basic safety information in situations that involve a risk to life."

This means apps in the Health & Fitness category, specifically those providing mental/emotional support via chat, journaling, or AI conversation, must include crisis protocol or they will be rejected.

What Causes Rejection

Based on documented developer experiences and Apple review decisions:

Missing Element	Rejection Reason Cited
No crisis resource screen at all	"App does not refer users to emergency services in life-risk situations"
Crisis resources present but unreachable	"Safety information not surfaced when needed"
Only hardcoded text with no detection	Usually passes if the information is accessible; detection is best practice not always enforced
Onboarding with no disclaimer	Sometimes flagged; unclear rule — include it to be safe
No way to call emergency services from within app	Flagged for some markets

Apple Safe Messaging Alignment

Apple doesn't formally cite "safe messaging guidelines" in the written guidelines, but review teams are known to apply the AFSP (American Foundation for Suicide Prevention) and SAMHSA safe messaging standards informally. Key rules:

DO provide crisis line numbers
DO use supportive, non-alarmist language
DON'T provide method details (how to hurt oneself)
DON'T use clinical language that pathologizes or stigmatizes
DON'T ask probing questions about method/plan (leave that to trained counselors)

Google Play Store

Google Play's policies are less prescriptive but effectively equivalent. Apps in the "Health" category must "provide information about relevant emergency services or provide basic safety information in situations that involve a risk to the life of themselves or others." Same requirement, same enforcement.

2. How Approved Apps Do It

Woebot

Architecture: Rule-based NLP (not free-form LLM generation). Conversations are scripted by clinical writers + NLP routes to scripts.

Crisis detection: Dedicated NLP classifier developed with clinical psychologists. Continuously monitored and tested for accuracy. Detects "potentially concerning/crisis language."

Response flow:

Classifier fires on user message
Woebot immediately pauses its normal therapeutic flow
Shows crisis message + external crisis resources
Does NOT attempt to counsel the user further on the topic
Explicitly states: "Woebot is not a crisis service"

Key UX principle: Woebot shows resources and steps back. It does not try to be the crisis responder. This is the correct model.

Detection approach: Keyword + classifier hybrid. Not dependent on LLM output classification.

Wysa

Architecture: AI chatbot + structured CBT module library. Listed by NYC 988 as a crisis-integrated mental health resource.

Crisis handling: When crisis language is detected, Wysa surfaces:

Local crisis hotline number (geo-targeted)
In some markets: direct 988 integration
Option to contact a Wysa coach (paid feature, not available in all markets)

Tone: Empathetic acknowledgment, then immediate resource surfacing. No deep clinical probing.

Detection: Combination of keyword matching + intent classification. Not purely LLM-based.

Calm / Headspace

Both are wellness (not clinical) apps. They:

Include crisis resources in their "in case of emergency" screen accessible from settings
Have minimal in-context detection (Calm's AI chat does have basic detection)
Their crisis screen is lighter — primarily a static resource page
Less sophisticated detection because they market as wellness, not mental health support

Implication for our app: Since our app is an AI reflection app explicitly addressing emotional wellbeing, we're closer to Woebot/Wysa in Apple's eyes. We need the fuller implementation.

Common UX Pattern (All Approved Apps)

User sends concerning message
         ↓
[Detection fires — keyword OR classifier]
         ↓
App pauses normal flow
         ↓
Displays supportive message (3-5 sentences max)
+ Crisis resources card (prominent, tappable phone numbers)
         ↓
User can:
  A) Call the crisis line (tel: link)
  B) Dismiss and continue in app
  C) [Optional] Go to resources page
         ↓
App resumes (or session ends, depending on severity)

3. Technical Implementation Options

Option A: Pure Keyword/Phrase List (Recommended for v1)

Approach: Maintain a list of phrases. Run substring/regex matching on every user message before it goes to the LLM. If match → trigger crisis flow.

Pros:

Deterministic — will ALWAYS fire when keyword appears
No LLM latency on detection path
Easy to audit and update
Works offline
App Store reviewers can verify it

Cons:

False positives (discussed below)
Misses paraphrase/indirect expression
Requires maintenance as language evolves

Recommended: Use this as the primary safety net. Optionally layer LLM-based intent detection on top for nuanced cases.

Option B: LLM-Based Intent Detection

Approach: After every user message, ask the LLM to classify intent with a structured prompt. E.g.:

System: You are a safety classifier. Analyze the following message for suicidal ideation or self-harm intent. Return JSON: {"risk_level": "none|low|medium|high", "explanation": "..."}.

Pros:

Catches indirect expressions ("I just want it all to stop")
Fewer false positives for obvious non-crises
Handles Russian/multilingual naturally

Cons:

Adds latency (second LLM call)
LLM can fail silently, hallucinate classifications
Not deterministic — can miss high-risk messages
More expensive

Recommended: Use as secondary layer on top of keyword detection. Trigger crisis flow if keyword OR LLM flags high/medium risk.

Option C: Hybrid (Recommended for production)

User message
     ↓
[Layer 1] Keyword matching (fast, deterministic)
     ↓ (if no keyword match)
[Layer 2] LLM safety classifier (async, catches nuanced cases)
     ↓ (if either layer fires)
Crisis protocol triggered

Implementation detail: The keyword check should run synchronously before sending to the main LLM. The safety classifier can run in parallel with the main LLM call — if it returns high risk before the LLM response is shown, intercept and show crisis screen instead.

Handling False Positives

A critical UX problem: the user says "my friend said they want to kill themselves" or "this song is about wanting to die." The keyword fires, but the user doesn't need crisis resources for themselves.

Strategies:

Clarifying question first (preferred for borderline cases):

"I noticed you mentioned something that might be difficult. Are you OK, or are you talking about someone else?"
- Only use for indirect/third-person phrasing
- Never delay resources if the message is direct and personal
Soft framing of resource screen:

"It sounds like this topic is on your mind. Whether this is about you or someone you care about, here are some resources that might help."
- Covers both cases without assuming
- Reduces friction if it was a false positive
Context window check: Before firing on "kill myself," check if preceding messages establish fictional/song context. Use LLM to evaluate context: "Is the user discussing their own distress or referencing something external?"
Graduated response:
- Exact first-person high-intent phrases → immediate crisis screen, no clarifying question
- Indirect/third-person phrases → clarifying question first
- Metaphorical expressions ("I'm dying of embarrassment") → filter out of keyword list
Escape hatch on crisis screen: Always give users a way to say "I'm OK, this wasn't about me" → returns to normal flow. Track these dismissals to improve detection.

What the Crisis Screen Should Show

SHOW:

Empathetic 2-3 sentence message (warm, not clinical)
Crisis line number(s) for the user's region (large, tappable)
"Text" option if available in their country
A clear "I'm safe, continue" button

DO NOT SHOW:

Detailed questions about method or plan (not your job)
Long disclaimers or legal text on this screen
Advertising or subscription upsells
Complex navigation
Scary clinical language ("suicidal ideation detected")
Guilt-inducing language ("you shouldn't feel this way")

Example Crisis Screen Copy:

"It sounds like you might be going through something really hard right now. You don't have to face it alone — trained counselors are available 24/7 to listen and help.

[📞 Call crisis line: 8-800-2000-122] [Free, confidential]

[I'm OK, continue]"

4. Keyword Lists

English — High-Priority Triggers (Direct, First-Person)

These should fire the crisis screen immediately, no clarifying question:

Suicidal ideation:
- "want to die" / "want to be dead"
- "kill myself" / "killing myself"
- "end my life" / "end it all"
- "not want to live" / "don't want to live anymore"
- "take my own life"
- "suicide" (when paired with first-person context)
- "can't go on" / "can't take it anymore" (flag for LLM check)
- "no reason to live" / "no point in living"
- "better off dead" / "better off without me"
- "planning to kill myself"
- "goodbye forever" (flag for LLM check — high false positive rate)

Self-harm:
- "hurt myself" / "hurting myself"
- "cut myself" / "cutting myself"
- "self-harm" / "self harm" / "selfharm"
- "burn myself"
- "want to bleed"
- "punish myself" (flag for LLM check)

English — Secondary Triggers (Indirect, Require LLM Confirmation)

- "it all stops" / "make it stop" / "everything to stop"
- "fall asleep and not wake up"
- "everyone would be better off"
- "no one would miss me"
- "tired of living" / "tired of being alive"
- "not here anymore"
- "disappear forever"

Russian — High-Priority Triggers

Суицидальные мысли:
- "хочу умереть" / "хочу быть мёртвым"
- "убить себя" / "убью себя" / "хочу убить себя"
- "покончить с собой" / "покончу с собой"
- "не хочу жить" / "не хочу больше жить"
- "жить не хочется"
- "лучше бы я умер" / "лучше бы умерла"
- "конец жизни" / "положить конец жизни"
- "суицид" / "суицидальные мысли"
- "прыгнуть" (combined with "мост", "окно", "крыша") — flag for LLM
- "выпить все таблетки"
- "последний день" (flag for LLM — high false positive)
- "не хочу больше существовать"
- "всем будет лучше без меня"
- "никому не нужен" + "умереть" (combined)
- "больше не могу" (flag for LLM)

Самоповреждение:
- "порезать себя" / "режу себя" / "порезалась" / "порезался"
- "причинить себе боль"
- "самоповреждение" / "самоистязание"
- "жечь себя"
- "ударить себя"

Russian — Secondary Triggers (Require LLM Confirmation)

- "всё останавливается"
- "устал от жизни" / "устала от жизни"
- "исчезнуть навсегда"
- "никто не заметит"
- "последний раз" (very high false positive — LLM only)

Implementation Notes

Normalization: Before matching, lowercase, strip punctuation, normalize Cyrillic character variants (е/ё, etc.)
Stemming: For Russian, use a simple stemmer or match multiple inflections explicitly. "убью/убить/убивать/убиваю" should all trigger.
Negative contexts to filter: Maintain a list of phrases that override keyword matches (reduce false positives):
- "в кино" / "в фильме" / "в книге" / "персонаж" — likely fictional reference
- "моя подруга/друг/мама" — may be about someone else → use clarifying question
- "вчера я смотрел" — past tense media reference
Minimum word count: Don't trigger on messages under 3 words (too ambiguous)
Rate limiting: If crisis screen was dismissed in the last 15 minutes for the same session, don't retrigger on borderline keywords (avoid feeling like harassment)

5. Crisis Resources by Country

Auto-Detection Strategy

Method 1 — IP Geolocation (Primary)

Use device locale/IP at app launch to set country
Store in user preferences
Update if user changes device locale

Method 2 — User-Selected Country (Fallback + Override)

During onboarding, ask: "Which country are you in?" with a simple selector
Stored preference overrides IP detection
Include "Other / International" option

Method 3 — SIM/Phone Carrier (Most Accurate, Privacy Tradeoff)

CoreTelephony (iOS) / TelephonyManager (Android) can return ISO country code from SIM
No user permission required for country code only (no phone number)
Most accurate but doesn't work for users without SIM (WiFi-only iPads, etc.)

Recommended: Combine SIM country code (if available) → IP geolocation → user preference.

🇷🇺 Russia

Service	Phone	Audience	Hours	Notes
Детский телефон доверия	8-800-2000-122	Youth + children	24/7	Free nationwide, бесплатно
Экстренная психологическая помощь МЧС	+7 (495) 989-50-50	Everyone	24/7	EMERCOM emergency psych line
Телефон доверия (Суицид)	8-800-220-8000	Everyone	—	Suicide prevention hotline
МЫРЯДОМ.ОНЛАЙН	124	Youth/children	—	Also has online chat

Primary number to show: 8-800-2000-122 (free, widely known)
Secondary: 8-800-220-8000 (adults, suicide-specific)

🇰🇿 Kazakhstan

Service	Phone	Audience	Notes
Телефон доверия	150	Youth/children	Free, national
111 Service	111	Everyone, women, children	Also via WhatsApp: +7 701 000 1404

Primary number to show: 150
Also show: 111

🇰🇬 Kyrgyzstan

Service	Phone	Audience	Hours	Notes
National Mental Health Centre	+996 312 881 618	Everyone	Mon–Fri, 9AM–6PM	Kyrgyz + Russian
Youth Helpline	+996 312 662 866	Children/adolescents	Daily, 10AM–4PM
Child Rights Defenders League	111	Youth	—
National Emergency	112	Everyone	24/7	General emergency incl. mental health

Primary number to show: +996 312 881 618 (with note: Mon–Fri 9–18)
24/7 fallback: 112 (general emergency)
Important note: Kyrgyzstan does not have a 24/7 dedicated mental health crisis line as of 2026. The 112 emergency line should be presented as the 24/7 option.

🌍 International / Other Countries

IASP Directory: https://www.iasp.info/resources/Crisis_Centres/
Use this as the fallback link when country is unrecognized.

Country	Number	Notes
USA	988	Suicide & Crisis Lifeline
UK	116 123	Samaritans
Germany	0800 111 0 111	Telefonseelsorge
EU general	116 123

Crisis Screen Country Logic

// Pseudocode
func getCrisisResources(countryCode: String) -> CrisisResources {
    switch countryCode.uppercased() {
    case "RU":
        return CrisisResources(
            primary: CrisisLine(name: "Телефон доверия", number: "8-800-2000-122", free: true),
            secondary: CrisisLine(name: "Телефон доверия (взрослые)", number: "8-800-220-8000", free: true),
            chat: nil
        )
    case "KZ":
        return CrisisResources(
            primary: CrisisLine(name: "Телефон доверия 150", number: "150", free: true),
            secondary: CrisisLine(name: "Служба 111", number: "111", free: true),
            chat: nil
        )
    case "KG":
        return CrisisResources(
            primary: CrisisLine(name: "Центр психического здоровья", number: "+996312881618", free: false, hours: "Пн–Пт 9:00–18:00"),
            secondary: CrisisLine(name: "Скорая помощь (круглосуточно)", number: "112", free: true),
            chat: nil
        )
    case "US":
        return CrisisResources(
            primary: CrisisLine(name: "988 Suicide & Crisis Lifeline", number: "988", free: true),
            secondary: nil,
            chat: CrisisChat(url: "https://988lifeline.org/chat/")
        )
    default:
        return CrisisResources(
            primary: CrisisLine(name: "IASP Crisis Directory", number: nil, url: "https://www.iasp.info/resources/Crisis_Centres/"),
            secondary: CrisisLine(name: "Local emergency", number: "112", note: "Or your local emergency number"),
            chat: nil
        )
    }
}

6. Legal Liability & Disclaimers

Does a Crisis Protocol Reduce Legal Risk?

Yes — significantly, but with caveats.

The case FOR reduced liability:

Due diligence defense: If a harm event occurs and you can demonstrate you had a crisis protocol, provided emergency resources, and followed safe messaging guidelines, your negligence exposure is substantially lower.
Industry standard compliance: Woebot, Wysa, and other approved apps set the standard of care. Not having a crisis protocol when they do = potential negligence finding.
App Store compliance: Rejection prevention is itself a risk mitigation — an app without proper safety features implies the developer didn't take safety seriously.
Good Samaritan provisions: In many jurisdictions, good-faith attempts to provide safety resources can provide partial legal protection.

Important caveats:

A crisis protocol does not make you liable as a medical provider — provided your disclaimers are clear
The disclaimer must be seen and acknowledged by the user
You must not make clinical claims (diagnosis, treatment) — "reflection app" framing is better than "therapy app"
Terms of Service should explicitly state the app is not a substitute for professional care

Required Disclaimers

Onboarding Disclaimer (must acknowledge before using app)

[Acknowledgment required — checkbox or "I understand" button]

This app is designed as a personal reflection and emotional wellness tool. 
It is not a substitute for professional mental health treatment, therapy, 
or crisis intervention.

If you are experiencing a mental health emergency or are in immediate danger, 
please contact emergency services (112 / 911) or a crisis hotline immediately.

[I understand and agree to continue]

Settings / "About" Page

Mental Health Disclaimer
[App Name] is a wellness and reflection app, not a clinical service. 
It does not provide diagnosis, treatment, or medical advice.

In case of emergency:
• Russia: 8-800-2000-122 (free)
• Kazakhstan: 150
• Kyrgyzstan: +996 312 881 618
• International: iasp.info/resources/Crisis_Centres/

For clinical care, please consult a licensed mental health professional.

Terms of Service — Required Clauses

Scope limitation: "The App is intended for personal reflection and general wellness purposes only. It is not a medical device, clinical service, or substitute for professional mental health care."
Emergency services: "If you or someone you know is in immediate danger, contact local emergency services immediately."
No professional relationship: "Use of the App does not create a patient-therapist or doctor-patient relationship."
Data and crisis: "In jurisdictions where required by law, [Company] may be required to report credible threats of imminent harm."
Limitation of liability: Standard; ensure it includes limitation for mental health outcomes.

Jurisdictional Notes

Russia: No specific law mandating crisis protocol in apps, but general duty of care applies. The Federal Law on Protection of Citizens' Health broadly applies.
Kazakhstan: Similar to Russia. Digital health apps are in a regulatory gray zone.
Kyrgyzstan: Least developed digital health regulation — general civil liability applies.
EU (if expanding): MDR (Medical Device Regulation) could apply if app makes clinical claims. "Reflection app" positioning avoids this.
USA (if launching there): FTC Act applies to misleading health claims. Avoid "treats depression" language.

7. Implementation Spec

Overview

The crisis protocol consists of 4 components:

Detection Engine — identifies crisis language in user messages
Crisis Screen — what the user sees when detection fires
Onboarding Disclaimer — shown once at first launch
Resources Page — always accessible in settings

Component 1: Detection Engine

Trigger points: Runs on EVERY user message submitted to the AI, before the AI generates a response.

Architecture:
  Message input
       ↓
  [Keyword Matcher] — sync, <5ms
       ↓ match found?
       YES → trigger crisis flow immediately
       NO  → 
             ↓
       [LLM Safety Classifier] — async, parallel with main LLM call
             ↓ returns HIGH or MEDIUM risk?
             YES → intercept before showing LLM response → trigger crisis flow
             NO  → show normal AI response

Risk levels:

CRITICAL — exact match on high-priority keyword list → immediate crisis screen, no clarifying question
HIGH — exact match on secondary list OR LLM returns "high" → immediate crisis screen
MEDIUM — LLM returns "medium" → clarifying question: "I noticed you mentioned something heavy. Are you OK?"
NONE — no match → normal flow

LLM Safety Classifier Prompt:

You are a safety classifier for a mental health app. Analyze the following user message for suicidal ideation or self-harm intent. Consider context if provided.

User message: [MESSAGE]
[Optional: Last 3 messages for context]

Respond ONLY with JSON:
{
  "risk_level": "none" | "low" | "medium" | "high" | "critical",
  "reason": "brief explanation",
  "is_about_self": true | false,
  "is_fictional_reference": true | false
}

Risk levels:
- critical: explicit statement of plan or intent to harm self imminently
- high: clear expression of suicidal ideation or self-harm desire, first-person
- medium: indirect expression of wanting to not exist, passive suicidal ideation
- low: general hopelessness, no specific harm intent
- none: no crisis indicators

Persistence: Log all MEDIUM+ detections to analytics (no PII, just event + risk level + session ID). Review weekly to improve keyword list.

Component 2: Crisis Screen

Trigger behavior: Replaces the normal AI response. The AI response is discarded.

Screen layout (mobile):

┌─────────────────────────────────────┐
│  [X] Close                           │  ← always present, but subtle
│                                      │
│  💙                                  │
│                                      │
│  You don't have to go through        │
│  this alone.                         │
│                                      │
│  It sounds like things feel really   │
│  hard right now. Trained counselors  │
│  are available to listen, any time.  │
│                                      │
│  ┌───────────────────────────────┐   │
│  │ 📞 8-800-2000-122             │   │  ← tel: link, opens phone dialer
│  │    Телефон доверия (бесплатно)│   │
│  └───────────────────────────────┘   │
│                                      │
│  ┌───────────────────────────────┐   │  ← only if country has it
│  │ 💬 Написать онлайн            │   │
│  └───────────────────────────────┘   │
│                                      │
│  ───────────────────────────────     │
│                                      │
│  [I'm safe, continue →]              │  ← smaller, secondary button
│                                      │
└─────────────────────────────────────┘

Copy guidelines:

Warm, not clinical
"Trained counselors" not "therapists" or "psychiatrists" (less intimidating)
"Listen" not "treat" or "diagnose"
No phrase "suicidal thoughts" on the screen — user knows what they said
Language matches app locale (Russian for Russian users)

After "I'm safe, continue":

Log dismissal (for analytics)
Ask clarifying: "I'm glad you're safe. Would you like to continue with what you were sharing?"
Return to normal chat flow
Do NOT re-trigger crisis screen in the same session for similar messages unless CRITICAL level

Component 3: Onboarding Disclaimer

When: Screen 2 of onboarding (after welcome screen, before first substantive screen).

Required: User must tap "I understand" or similar to proceed. Cannot swipe past.

Content:

A note before you begin

[App Name] is here to support your reflection and emotional wellbeing.

It's not a replacement for professional care. If you're ever in crisis 
or having thoughts of harming yourself, please reach out to a crisis 
line or emergency services — they're trained to help in ways I'm not.

[I understand — let's get started]

Store the acknowledgment: Save timestamp of acceptance to user preferences. Re-show if Terms of Service version changes.

Component 4: Resources Page (Always Accessible)

Location: Settings → "In an emergency" or accessible via a subtle "?" icon in chat header.