Quality Assurance Capability

Replacing a dying Windows program that checked map data for the U.S. government's defense mapping agency with a React app built on the government's own design system

My role

Lead product designer

Employer
World Wide Technology
Client
National Geospatial-Intelligence Agency
Years
2018 to 2020
Team
One of three engineers building the screens
  1. Led the UX of the program that checks map data before it goes to NGA, and built its screens in React.
  2. Set up the weekly rhythm: one demo showing what we learned from users, the prototypes and the finished features, with research every two weeks.
  3. Led the design of, and helped build, more than 40 pieces the government design system was missing, each with instructions and automatic tests.
  • Days→hours to check a batch of map data, as contractors told us; nobody timed it
  • 2,000+ map files checked in the first month, by the delivery team's own counts
  • 40+ screen parts the government design system lacked, built in React with tests
  • WCAG 2.1 AA accessibility standard met, checked by us with tools and a screen reader, no outside auditor

Summary

  • Contractors in NATO countries Had to prove their map data was free of mistakes before sending it to NGA; they ran the tool on secure laptops in the field.
  • Mapmakers Answered to their commanders; nobody running the project thought of them as the customer.
  • IT staff in closed networks Installed updates from removable drives, digitally signed so they knew the updates were genuine.
  • Three engineers Built the screens; I was one of them, and led the UX.

Contractors in NATO countries had to send map data to the National Geospatial-Intelligence Agency (NGA), the U.S. government's mapping agency for defense, with no errors in it. The only program that could check it was written in Visual Basic, and Microsoft was ending support for the Windows it needed, so contractors couldn't install it at all.

We replaced it with a web app wrapped in a desktop installer, so it runs on laptops kept off the internet on purpose. A check takes four guided steps, with no command line: drop the folder in, confirm it's complete, run the detailed checks, and download the report. I led the UX, built screens as one of three engineers, and set the weekly habit that put what we heard from users straight into the work underway.

The results screen. A headline reads '44 things to look at' with a chip for 2 blocking conditions and the run's provenance: MGCP 4V4.5, cell N50E016, Poland / Czech border, finished 14:22. Below left, an OpenStreetMap basemap of the cell carries large red markers for blocking findings and smaller blue ones for advisory, inside a dashed cell boundary. Below right, the six conditions are listed blocking-first with their counts.
Problems on a map. The map shows where each problem is, and the list puts the must-fix ones at the top.

Context

The old checker was written in Visual Basic, an old Microsoft programming language, and Microsoft had announced it would stop supporting the version of Windows it needed. Contractors couldn't install it at all, which put the whole requirement at risk. Using it meant clicking through one pop-up form after another while following a checklist from a binder six inches thick. Every submission took days.

Federal rules also said the new tool had to follow the U.S. Web Design System (USWDS), the government's design system. In 2018 that was little more than a style guide and a template file, meant for web pages, not for a working program.

Research

A plan diagram of how the checker reached its users. Outside, a signed release is copied to a removable drive. The drive is carried by hand through the installation's checkpoint. Inside, on a closed network with no internet, IT staff check the signature before installing it on offline laptops, where contractors check their map data. A dashed path marks the research team visiting on site to watch the work.
Crossing the air gap. Releases reached offline laptops on signed removable drives, and we went on site to watch the work.
A plan diagram of how the remote interviews connected. The research team's laptop takes a Common Access Card in a card reader, and the card's certificate and PIN prove who is connecting. The connection runs as an encrypted tunnel across a shared federal network, where a gateway checks the certificate before it passes. Inside the installation, the people doing the work join a video session. The checker's offline laptops are marked as never on this connection.
Interviews over a CAC-protected line. Between visits we interviewed people by video, over an encrypted connection the card's certificate unlocked.

The research was field research. We went to military installations to watch the work where it happens, and interviewed by video the people our client contact pointed us to: the ones already doing this job the hard way, because they know best what the tool really has to do.

After that we did research every two weeks. Each weekly demo showed three things in one sitting: what we had learned from users, the prototypes we were designing, and the features we had finished. Every prototype had to answer something we had heard from users, and every finished feature had to match the prototype we agreed on.

Design

The new program is a web app wrapped in a desktop installer, so it installs like any desktop app and runs on computers cut off from the internet and from any company network. A check takes four guided steps, with no command line: drop the folder in, confirm it's complete, run the detailed checks, and download the report. Updates came on removable drives, digitally signed so the IT staff inside those closed networks knew they were genuine.

The government's design system had nothing for a working program, like pop-up windows, step-by-step wizards or data tables, so we built more than 40 of those pieces ourselves in its style, in React. We wrote instructions and automatic tests for each one, and wrote accessibility into every feature's to-do list from the start instead of checking at the end.

The front door. A thin banner across the top reads 'An application for data producers.' A hero reading 'Check your data before it ships' sits above Start a check and See a sample report, beside a sample-report card for cell N50E016 showing a real basemap with red and blue markers and a condition list: point not inside area 6, area feature not transferred 12, complete duplicates 4. Below, three numbered cards explain how a check works: pick a schema and upload, we check the payload then the KERs, you get a map of what to fix. The footer carries accessibility, support and conformance links and states the tool is built to WCAG 2.1 AA and EN 301 549.
The front page. The front page explains the steps of a check before you upload anything, and lets you open a sample report first.
The payload check failed. A red rule tops the card, which reads 'Three things are wrong with the folder' and explains that KER checks cannot run until the payload is correct. Three faults are listed: a missing TransportationGroundCrv layer, an unexpected EPSG:3857 projection where the protocol expects EPSG:4326, and two files not part of the protocol. The only actions are Fix and upload again, and Download the payload log.
A hard stop. If a layer is missing, the projection is wrong or stray files are in the folder, the tool stops here and lists exactly what's wrong. There's deliberately no button to skip ahead to the detailed checks.
A blocking condition opened: Point not inside area, KER 2.4.1, 6 features affected. Plain language leads: a well head recorded outside its facility boundary, so anything grouping points by containing area will miss them, followed by 'How to fix it', which directs the edit to the production database. A map shows where it happens, and a table lists affected feature IDs, layers and positions, showing 3 of the 6.
One must-fix problem. Each problem is explained in plain English, with how to fix it; the rule number is a label, not the headline.

More screens

The payload check passed. The card reads 'The folder has everything it should' and summarizes 14 layers, 61,208 features, EPSG:4326. A table lists each layer with its feature count and a Complete status, above a Run KER checks button.
A pass that shows its work. When the folder passes, it lists the layers it found with their feature counts, so you can tell a real pass from a check that never ran.
KER checks running against 61,208 features and 96 restrictions. The progress note names the restriction class being evaluated: checking KER 2.4.1, containment, and a line below states the results will be held under Running checks.
Running the detailed checks. The slow step names the rule it's checking right now, not just a percentage.
The download step for cell N50E016, 44 features, MGCP 4V4.5. Format options offer .shp only, .csv only, or .csv and .shp, each with a note on what it is for; scope offers all 44 features or blocking only. The download button names the chosen format, and a note restates what the current selection will produce.
Choosing what goes in the report. You pick the file type and what to include, and the download button repeats your choice instead of hiding it in the options.
The Running checks list. Four jobs are shown with cell, protocol and start time, each with a plain-text status and an action matched to its state: View progress for a run at 62%, View for one queued, Open report for a finished run with 27 features, and See why for one whose payload check failed. A note states that checks keep running if the tab is closed and finished reports stay for 30 days.
Checks that keep running. Checks keep running if you close the tab, and each one has a button that fits where it's at: view progress, open the report, or see why it failed.

Outcomes

Contractors told us checking a batch of map data went from taking days to taking hours, so they got their data to NGA sooner. Nobody kept a before-and-after timing, so that is their account, not a measurement. The tool checked more than 2,000 map files in its first month, by the delivery team's own usage counts. It met the federal accessibility standard (WCAG 2.1 AA); we checked that ourselves with automatic tools and a screen reader, and no outside auditor reviewed it.

The lesson that stuck with me wasn't technical. What got leadership on board was pitching the component library, our automatic scripts and our way of shipping updates as ways to keep the project from going wrong, not as UX extras. That also protected the plan when new requirements showed up late.

  • 40+

    Missing design system pieces, built in React with instructions and automatic tests.

  • Weekly

    Demos of what we learned from users and the finished features, with research every two weeks.

What my delivery lead said

  • In all this, Bobby was indispensable. Not only was he skilled enough to jump into the look-n-feel and execution of the project, but he was extremely well-versed in human-centric design and helped mentor many of our consulting teammates in the practice as well. He had a strong urge to learn what he could from everyone on the team and to share what he knew. Overall, he was a great asset! He was a great teammate and I would love to work with him again in the future.

    Matt Todd Delivery Lead

enesru