Method report

What patients came for, and why the site could not tell us

A discovery sequence run for an independent, multi-specialty physician group, beginning October 2014. Four instruments, sixteen months before anything shipped. Published here as method: the client is de-identified and the competitive set is described by role rather than by name.

Participants came from Lead User Sampling, the technique von Hippel described in Management Science in 1986. Six patients sat for interviews, three already familiar with the group and three not. More than eighty people answered a blind single-answer survey. Five stakeholders gave contextual interviews with a participatory exercise inside them. Six cognitive walkthroughs covered four direct competitors and two indirect.

The shape of it

Discovery ran from October 2014 and put nothing into production for sixteen months. Sketch prototyping started on day one and iterations ran in staging once the discovery report was complete. Only the release waited. Three instruments did the work. A survey went to visitors on the live site. Six one-hour patient interviews fed a journey model. Five contextual interviews with the group leadership were scored for agreement.

Four instruments

Each one answers a question the others cannot. That is why we ran four rather than the biggest one twice.

Intercept survey

80+ blind single-answer responses, served to visitors mid-visit on the live site

Answers: Why people came, in their own words, ranked
Cannot answer: Why they left, or what order they wanted things in

Patient interviews

Six one-hour sessions, three familiar with the group and three not

Answers: The sequence of questions a patient works through, and where it stalls
Cannot answer: How common any one pattern is

Stakeholder contextual interviews

Five business leaders, with a participatory sorting component

Answers: What the organisation wants, and where it disagrees with itself
Cannot answer: Whether any of it is what patients want

Cognitive walkthroughs

Four direct competitors and two indirect, walked task by task

Answers: What the category had settled on, and what nobody had done
Cannot answer: Whether the category was right

How participants were recruited

Lead User Sampling von Hippel, Management Science, 1986

Recruit people already trying to solve the problem with whatever tools exist, instead of a representative cross-section. Lead users have worked around the failure many times. They can describe it precisely. That is why a six-person sample is productive even though it cannot be representative.

What people came for

One question, one answer, asked mid-visit on the live site. The question read: “Please select the reason for your visit”.

Reason for visit	Share	95% CI
I need to find a physician	25.3%	−8.1 / +10.3
I need access to my health records	24.1%	−7.9 / +10.2
I need to find a specialty practice	12%	−5.4 / +8.7
I need to know who sent an invoice	12%	−5.4 / +8.7
I need info on the group	10.8%	−5 / +8.5
I need to verify my insurance coverage	9.6%	−4.7 / +8.2
I need to research general health info	3.6%	−2.4 / +6.5
Careers // Job	2.4%	−1.7 / +6

The top two rows are the finding. 49.4% of visits were someone trying to find a physician or reach their records. That figure is just two rows added together. Records access is a quarter of the traffic, yet it usually sits behind a portal link in the footer.

The journey the interviews described

Synthesised from six one-hour sessions. Note which phases are linear and which are not: a phase that loops cannot be designed as a funnel.

Research

Non-linear

Primary shopping phase: ongoing and non-linear

Finding doctors recommended to me who accept patients & my insurance

Internet search
Physician referral
Friend or relative
Insurance directory
Third-party doctor listings
Print, radio, television, billboard
Community event
Find a Doctor

Scheduling

Linear

Primary purchase phase: time-based and linear

Scheduling an appointment with a healthcare professional

Request Appointment form
Click to call
Practice front desk

Pre-Visit

Non-linear

Waiting phase: time-based but non-linear

Waiting for and completing intake paperwork

Get Intake Form
Patient Portal
Practice by phone

Visit

Linear

Primary service delivery phase: time-based and linear

Consulting with the doctor about my health treatment needs

In person, with the clinician

Post-Visit

Non-linear

Primary loyalty phase: time-based and non-linear

Scheduling follow-ups. Paying bills. Updating my loved-ones

Pay My Bill
Health Records
Patient Portal
Follow-up by phone
Family and carers

What the organisation wanted

Five contextual interviews, sorted into MoSCoW bands, each showing the share of participants who placed it there. Recording the percentage keeps disagreement visible instead of averaging it away.

MUST

Improve awareness & ranking providers in search engine results 100%
Improve integration of group brand strategy 100%
Improve awareness & evaluation of patient-related value propositions 75%
Improve cross-marketing opportunities 75%

SHOULD

Improve online appointment scheduling 75%
Improve site resources/content engagement 75%
Improve awareness & evaluation of provider recruitment value propositions 75%
Improve patient understanding of costs/payments for services & bill readability 67%

COULD

Improve provider recruitment and retention rates 100%
Improve awareness of community projects 75%

WON'T

Improve health information portals 100%
Improve online appointment scheduling 25%

Where they disagreed

Improve online appointment scheduling appears in more than one band. That is disagreement on record. It is the most useful thing the instrument found, because it showed the fight before anyone spent money on it.

What the category had settled on

We walked six sites task by task: four direct competitors, other physician groups and health systems in the same metro area, and two indirect, a national insurer's provider directory and a national appointment marketplace. Captures came from the Internet Archive at the time of the walkthroughs and again three years later.

None of the six had any structured data describing physicians, practices or procedures, in either era. Not one entity was typed. The category had settled on a directory a search engine could only read as prose. That left typed entities open as a position rather than a hygiene factor.

Competitor names are withheld. Naming six providers in a single metro area identifies the client by inference.

What this does not establish

Stated in the deliverable at the time: the sample is not representative of the general population, and this is not a statistically valid quantitative study. It is a contextual exploration meant to surface issues that lead users are especially articulate about.

Three sections of the original deliverable are not reproduced here, because they are not in the record this site is built from:

The six cognitive walkthrough findings; the record keeps only the fact they were run
The patient interview write-ups behind the journey model
What the deliverable concluded, beyond what it reported

No post-launch measurement was agreed before the work started, and none was retained afterwards. The strategy is documented and its effect is not. Every engagement since has started by writing down what would have to be true for the work to succeed, and how anyone would know. A metric chosen afterwards is a metric chosen to flatter.

Provenance

Collected on the live site and in scheduled sessions between October 2014 and March 2016, and reported with confidence intervals. Figures are quoted from the discovery deliverable of 8 January 2015.