The Challenge

When employees are grabbing a quick cup of coffee, we have their attention for a span of approximately 10 seconds. We asked ourselves how we can best leverage this time span.

The Director of our department decided to purchase a digital display to to fit nicely behind our coffee machine at work. The project was very open ended and did not come with any set of requirements or goals.


The Vision

We developed digital signage that allows employees to engage by posting content such as an event or a Happy Birthday greeting to a co-worker. This content is displayed along with news headlines pulled from Twitter, videos, other curated content from the admin dashboard.

The purpose of this app is to:

  • Create a display that provides relevant and useful information for users.

  • Provide engagement and awareness toward KP related topics.

  • Develop a scalable product that can be easily implemented in other locations (even if it’s not in front of a coffee machine).

  • Develop a product that showcases and stretches our abilities.

My Role

Being a part of a small team of designers and engineers, I get to wear many hats.



The director of the department came to us with a monitor and full autonomy. We quickly dove into uncovering pain points and brainstorming. 


I brought the wireframes and mockups to life with animation through After Effects. It gave us a more realistic view of our product.

user testing

To ensure we were developing something useful, I made sure we conducted user testing throughout the entire process.


Meet the rest of the team:

2 Engineers

Lessons Learned

This wasn't the first digital display I've worked on. I've made plenty of mistakes and wanted to make sure I applied everything I learned to this new project.


Lessons learned from a previous project:

  • If you do not consider maintenance at the beginning of the project, it will come back to haunt you.

  • Content will become old and uninteresting very quickly.

  • Other departments/teams will ask if you can build one for their department. (This is where the goal of scalability came from.)




The users of the coffee machine are primarily going to be users of the coffee machine. Our primary users are those that sit on the same floor as us, the 5th floor.



We looked a few other companies and the type of content they were displaying on their digital display.



On the Left: Some initial testing to see which animation worked best. We used masking tape to indicate where a user would typically stand. 

In the Middle: Testing some higher fidelity prototypes. Making final adjustments. Our soft launch of the display. 



There are 4 different levels of information going from the most broad (1) to the most specific (4). 

1: Global - general information or news pertaining to healthcare, technology, etc.

2: KP - general information pertaining to Kaiser as a whole

3: Department - information pertaining to specific departments within Kaiser.

4: Individual - information pertaining to individual employees.



We knew we wanted to utilize a card system and so we went many iterations of animation to ensure the optimal experience. I also made sure I talked to engineering frequently to discuss feasibility. 


card layout

Now that we figured out how the card were going to move, we moved on to figuring out the layout of them. 


visual design

The monitor displays a series of cards that pulls content from Twitter, YouTube and custom made content. We went through several iterations of the visual design of the card while making sure we adhere to the department's style and aesthetic. 


user testing

We leveraged online tools and hallways testing to collect feedback and data.



Having come up with several different variations, we did some user testing to narrow it down to 3 possible choices. From those 3 choices we then conducted some hallways testing to narrow it down the the final design.



We want people to be excited about our product. In order to validate whether we were going in the right direction or not, we announced features that hasn't even gone into production. For example, we announced that we would feature the weather but it turns out no one cares about the weather. (Maybe because this is Southern California, where the weather never changes.)

We went back to the drawing board. That's when we discovered a pain point. Kaiser has an awesome shuttle system but the schedule is on this piece of paper that everyone always loses. With this in mind, we created a countdown for the next shuttle.


VISUAL DESIGN - final design

We created 5 different variations, one for each card category which includes: Social Media, Events, Videos, Shoutouts, and Quips.


the delightful factor

Not only do we pull live feeds from sources such as Twitter and YouTube, we created our own content. Our display is located behind a coffee machine so it's only appropriate that we showcase the types of drinks it makes through fun animated GIFs.

We realize most people already know what sparkling water or tea water is, but people still get a real kick out of it. Instead of the display and machine being two separate entities that happen to be in close proximity, this helps us create a relationship between the two things.

Nitty Gritty

If we want this display to function predictably and exactly the way we wanted it to, we needed to establish plenty of rules for it.

a few rules

Each card category has a few different rules and levels of access.


scheduling algorithm

How does the display know which card to pull? We knew we wanted the experience remain as diverse as possible so we developed an algorithm accordingly.

Next Steps

We're still hard at work perfecting, iterating, and exploring new ideas for this project. Our next big step is to create an admin panel that serves as a tool to allow users to upload and manage content.


wireframes - ADMIN PANEL

In order to create a scalable product that's also easily accessed by users, we need to develop an admin panel. Below are a few of the early iterations. 


mockups - admin panel

This page allows users to add/edit their own events. 


The Challenge

For typical patient with diabetes common tests include: Hemoglobin A1C and Creatinine. The results of these tests can be difficult to decipher due to its lack of context and color. If patients had a stronger understanding of their overall health, they would be better equipped to make better decisions about their health.

The Vision

The goal of this web app is to: 

  1. utilize human-centered design to develop a system of displaying lab results that makes it easier for patients to interpret

  2. create universally accessible data that can be accessed on desktop, tablet, and smartphones but also as a printout

  3. present patients with actionable items according to their specific needs that are both short term and long term

My Role

I worked closely with another UX Designer and a Physician to develop a proposal and prototype to pitch for funding.


user testing

I collected both qualitative and quantitative data to aid design decisions and ensure we were moving in the right direction.

solution brainstorming

Meetings with physicians, designers and engineers to to come up with ideas on how to best create this interactive product.


I went to numerous rounds of wireframing to optimize and develop the best user experience possible.


Meet the rest of the team:

UX Designer, KP Physician


There are many problems we want to solve within the healthcare space but during the discovery phase we narrowed in on specific issues we can tackle.



I looked into existing products on the market to see what they did well and what some of the concerns were.



Consulting the physician, we came up with several different groups of users.



Although we had many potential issues we could solve, we decided on one that suited the services of our department. We focused on 3 main topics: interactivity, connecting patients with healthcare professionals, and developing personal action plans for patients.


After honing in the problem we wanted to tackle, we ran with it.



After assessing several different existing products we began to understand what type of content we wanted to have while leveraging all the resources KP has to offer.


1st iteration

We developed the first iteration very quickly to get feedback on from physicians and managers.The consensus was to continue to push the digital experience and focus on the personalize action plan for each patient. 



After gaining a better understanding of what the physician is envisioning we went through another round of wireframes, this time spending more time fleshing out the details. 

3rd iteration

At this point I began to work with another UX Designer to gain feedback and further develop the wireframes into mockups and eventually a prototype.



To make sure we were moving in the right direction I conducted some user testing.



Do users have an easier time interpreting bar graphs over line graphs? In this scenario, the accuracy remained the same. Regardless of the type of graph we presented, users were able to understand and interpret the graph. The biggest difference between all 3 solutions is the amount of time it takes for the user to get to that state.


overall health score

Although the standard deviation for the letter grade system was larger, users were able to develop an interpretation of their score quicker.


past results

We not only want to give users a more holistic, birds-eye view of their health but nitty gritty specifics as well. We aren't looking to reinvent the wheel here but we did want to try and optimize the time it takes for users to interpret and understand their results. Overall, we were able to improve this section by 17.5 seconds.


4TH ITERATION (final iteration)

Using all the feedback and testing, we made some changes and developed an optimized version.


I then turned it into a clickable prototype that would help aid us in pitching the idea to business partners.



This project was created during a hackathon sponsored by both Kaiser Permanente and Microsoft.


The Challenge

Sharing the real perspective, the whole story, a day-by-day breakdown of a patient’s lifestyle doesn’t always occur in a healthcare setting. So how is a patient’s story told to get the best care possible? For the 72 year-old Chinese woman who can’t speak English, for the scared man who conceals the severity of his symptoms, for the manipulative patient who tries to increase his dosage - we need to create a web application as a tell-all for what goes on outside of the doctor’s office.

The Vision

Alluringly sleek, intuitively straightforward, and invitingly useful, Round Robin goes beyond just a pretty interface. To open the doors to the magical world of 360-degree evaluation of a loved one, all it takes is for the primary caregiver (appropriately dubbed Batman throughout this submission) to create an account for the patient. Batman can invite friends and family to participate by using the super special invitation code generated specifically in reference to each patient. The physician is loaded along with supporting medical staff. Once someone logs in, they instantly see a dashboard of existing data and can easily add a new entry (aka new feedback). Busy medical professionals (especially those superstar doctors) can simply answer three general questions about how a patient is doing and optionally provide a quick note for each answer. For the loved ones with more time, someone who loves providing extensive narrative accounts, or someone who loves using beautifully coded forms, they can answer additional questions catered more specifically to the patient and condition. 

True to the name, the patient's loving circle of care uses a round robin style of reporting to provide a 360-degree view of the patient's patterns. Well, for the medical staff, its 360-degrees because they see all submitters and all content. Friends can’t see anything beyond their own submission. Batman fully sees what the medical staff has written, and can only see which friends and family members have submitted (but not content). Perhaps that means Batman has a 270-degree view. After all, no friend wants Mrs. Wilson to hear about their shenanigans with her son.

My Role

As a part of a team at a hackathon, this project was highly collaborative. 


user research

Due to the tight constraints of time of a the hackathon (48 hours) user research was condensed down to consulting with a physician quick interactions with other healthcare members.


At this stage we realized there were many features to had to cut in order to make this project feasible. With the help of a physician we were able to trim it down to a MVP.


While the developers took the wireframes and started building the product, I created mockups to further enhance the usability of the product.


Meet the rest of the team:

2 Engineers, Business Consultant, KP Physician


At the hackathon many people pitched ideas surrounding healthcare where we formed well rounded groups.



As we formed a group encompassing a consultant, a ux designer and 2 engineers we tossed ideas back and forth while gaining feedback from different physicians.



As we ideated and discussed what the product would be we quickly jumped into action and first laid out the wireframes. 



As a group we laid out the different screens and flow of the app.



As the engineers got started on building the underlying structure based on the wireframes, I went forward to create mockups.


This impromptu team has pulled together this project completely from scratch in less than 24 hours. Once well-rested and with less strenuous time constraints, we’re taking this idea back to Kaiser. We’ll attempt to implement it with existing care coordination platforms like ParkinsonNet and integrate it into Kaiser’s strong IT and clinical infrastructure. We’re KP-IT gurus, so that won’t be a problem. This app can bring together patient-based care, disease management, and a circular support group.

Planned Parenthood

The goal of this design sprint was to devise a future vision for a new front-end experience for Online Appt Scheduling that integrates with Planned Parenthood Online, as a foundation for future workshops & discussions in Q3 2017.

Screen Shot 2019-01-21 at 3.36.30 PM.png

Affinity mapping

After hearing stories on current issues and understanding the problem space we brought out the post-its to jot down possible solutions.

Screen Shot 2019-01-21 at 3.37.14 PM.png

Solution sketching

Next we reviewed all the ideas and voted on the best options both as individuals and as a team.

Screen Shot 2019-01-21 at 3.38.25 PM.png

Product visioning

At the very end we collectively created a future vision along with documenting assumptions we want to later validate.

The Challenge

Dunbar's number is a theory which states humans can only comfortably maintain 150 stable social relationships. Following this law, workplaces that exceed 150 people have a higher risk of becoming an anonymous workplace. The Southern California Business Information Office (SCAL BIO) has over 200 employees. This posed a problem. To maintain a sense of connectedness, we put a digital wall of faces in our lobby. The display in the lobby rotates through faces of SCAL BIO staff, and shares something positive about each employee.

The Vision

The keywords were clean and simple. Let the picture of our staff shine and encourage learning about our people. We had the employees answer the following 3 questions: "I like to", "I'm good at", and "I want to". The answers to these questions were combined with custom head shots to create a visual that consisted of a mosaic of animated tiles. The end result was a stunning center piece for our lobby that allowed us to promote our people and help us all feel a little more connected. 

My Role

Under the guidance of a creative director I not only worked on the user experience but also the visual design.


ui design

I developed a layout that took into account human behavior and worked around any technological constraints.


Utilizing my previous experience as a photography intern I held photoshoots for the staff that remained consistent but still allowed for their personality to shine. 

visual design

I created a set of icons to help visually represent the various statistics and information that's featured on the facewall. I also helped layout the typography and planned out the animation.


Meet the rest of the team:

Creative Director, Design Intern


Keeping in mind the purpose of the digital display, we explored different possibilities.



We utilized SurveyMonkey to create a survey that asked employees questions such as: "What is your favorite music genre?".


We explored different solutions such as touch screen monitors and utilizing motion gestures.


As we established what the limitations were and the types of monitors we would use we mocked up potential layouts. 



We established a look and feel through creating a color scheme, icons, etc. We also made sure to survey each person we took a photo of.



All the photography was done in-house in various conference rooms.



The animation of the facewall was created through After Effects. 


The final product was displayed across 6 monitors in a lobby.


final design

The video is played on loop and rotates through profiles of different people. 


As employees join and leave the team the facewall also gets updated to stay accurate. This happens on a biannual basis.

The Challenge

Kaiser Permanente is a large organization. KP Southern California alone serves over 3.7 million members, has 14 medical centers and 198 medical offices. In order to provide high quality healthcare we have a lot of projects, ideas, and pilots. Keeping track of all of these projects has proven to be difficult.

The Vision

The goals of this app is to:

  • Track projects by allowing users to upload new projects, edit projects, and find projects.

  • Help users more easily filter their views and reports based on their individual needs.

My Role

As the lead UX designer for this project I took on various roles that included usability testing and wireframing. 


usability testing

Since the users are scattered across the region it was vital that we took advantage of online usability testing.

Wireframing/prototyping/visual design

The wireframes and mockups were created through Adobe Illustrator and Axure.


meet the rest of the team:

2 Engineers, 1 Product Owner, 1 Scrum Master


Before going into developing this product we looked at the pitfall of our current solution.



Users can be split up into two main categories, those that do and don’t work for Kaiser.

competitive analysis

While looking into possible solutions we looked into applications such as Formstack.

Microsoft SharePoint Formstack
Purpose of the site:
This was the solution that was already in place. A tool to share, organize, and discover information.

Competitive Advantage:
• Store and access information securely.Integrate with other Microsoft products more easily.
• Changes can be made by non-developers.

• Usability issues.
• Does not create meaningful reports.
Purpose of the site:
Online form builder that enables users to create online forms, including surveys, contact forms, and event registration.

Competitive Advantage:
• Drag and drop, flexible form builder.
• Follows standards of the Americans w/ Disabilities Act, Section 508, WCAG 2.0
• Provides analytics / reports.

• Expensive to get an Enterprise Plan.
• Project filtering system is lacking.


We sat down and created a timeline for design and development to create a working prototype.



As a team of project managers, designers and engineers; we built the entire application in-house. 


high level workflow

Since we already had solution in place (Microsoft Sharepoint), users already had a mental model of the system. We did not alter the overall workflow too much to match that mental model, we also tried to keep terminology/taxonomy the same to ensure there is some level of familiarity. 



When trying to determine the MVP, we map out features according to their value and complexity. Complexity is directly correlated to the number of hours it would take to develop.



These are of course not all of the wireframes produced but just a few key screens. 

These were produced in Adobe Illustrator.


wireframes - detailed view

The filter function was incredibly important to our product and we wanted to get all the details just right. We wanted to answer questions such as: How many information is really necessary to show? How much information is too much? What happens when that piece of information gets too long?



These are not all the mockups produced, but just a few key select screens. 

These were created in Adobe Illustrator.




Along the way we continually tested the product to ensure that we were on the right track. Before it was even launched into alpha and beta testing phases, many of the issues were sorted out.

We also had a built in bug reporting / feedback system that allowed us to continuously gain feedback in areas that needed improvement. 

We also host bi-monthly meetings where we gather stakeholders and users to discuss new features and gain feedback.

Detailed View

A deeper look into some of the features.


before and after - dashboard

Left: Our old solution. Right: Our new custom built solution.

One major complaint that we had was that users we not able to see enough content on their screen at once. With this in mind we wanted to really optimize the space we had.

BEFORE AND AFTER - Filter system

Left: Our old solution. Right: Our new custom built solution.

We have over 50 questions in our intake form. Every question is also a column on our dashboard. In Sharepoint it was difficult to manage and re-order all these column so we came up with a new system (on the right) that allows users to simply drag and drop. It made it more intuitive and easy to use.


Left: Our old solution. Right: Our new custom built solution.

We have over 2,000 projects in our system so it's important to allow users to very quickly sort through these projects. In the old solution, any keyword search took the user to a new page with the results. It wasn't an optimal workflow and slowed down users so we implemented a live search (on the right).

Documentation / Collaboration

The other designers (includes the creative director & senior ux designer) mostly work off-site so it's even more vital to be able to produce effective documentation.

for other designers

This is the type of documentation I produce and present to other designers. Although the content varies, the overall flow remains consistent. 

It always starts with the problem we're trying to solve followed by 1 proposed solution. The following page provides research & data that backs up this solution. Lastly, I include any alternative solutions.

FOR engineers

An example of the type of documentation I would prepare for engineering. Light on the annotations since we sit directly next to each other and collaborate on a daily basis.


It’s been over 6 months since we first launched and since then we have:

  • gained over 200 users

  • house over 2,000 projects

  • service 15 major regions (in S.California)

Next Steps

As we finish phase 1 of this project we will continue to iterate and move forward to phase 2 and add more features while taking into consideration any learnings. 


Admin Panel (mockup)

Admin panel

For the past sprint I worked on designing an admin panel that would allow our business consultants control over some aspects of the program (such as deleting users and editing the project form) without having to ask the engineering team. Because there are other groups at our company that are interested in the product, in the long run, it helps us create a more packageable / scalable product. 


Starbucks Coffee Machine


Assessing the usability of a coffee machine through constraints and affordances.







People have such wildly different tastes in music; from person to person but also for any one given person. Apps such as Spotify allows users to curate and fine tune playlists but it can be challenging to find similarities across multiple people. Current solutions and workarounds to this problem include listening to something more generic such as the radio or pre-made playlists.


Develop an easy way for users to find and play music in the car that would make everyone happy. Increase satisfaction in what the music that gets listened to. Decrease the time it takes to find something to listen to.



The idea was originally meant to find overlapping taste in videos.


To better understand how other people find solutions to this problem we surveyed 30 people through Amazon Mechanical Turk. We started with a few screener questions to find people that would be potential users for this feature. This included collecting data in regards to age, gender, what the listen to in the car (we would have thrown out anyone that responded with "nothing"; we've kept data that responded with anything else), and if/how often there's someone else in the car with them.

In order to measure the efficacy of the proposal, we first established a baseline to serve as a benchmark for task completion time and level of satisfaction. (Note: For the sake of time and effort we've opted to ask users how long they think it takes rather than directly timing users.)


We asked users: "How long would you say it takes to find music everyone in the car agrees to listen to."

1 is very quick; 5 is very long.

We calculated the confidence interval with a 0.05 level of significance. We can be 95% confident that the population rating on this question is between 1.742 to 2.391. (TLDR: people felt picking music was relatively quick.)

We asked users: "How satisfied are you with the music that gets selected as a group in the car?"

1 is not at all satisfied; 5 is very satisfied.

We calculated the confidence interval with a 0.05 level of significance. We can be 95% confident that the population rating on this question is between 3.171 to 3.962. (TLDR: people generally pretty satisfied.)

We asked people how they decide what to listen to in the car with others -- there were a few re-occuring themes. Here are a few verbatim quotes.

“Based on time of day and mood.”
”I leave it up to them.”
”Take turns.”


We'd first find songs that are exact matches for songs, artists, or albums. Next we’d expand the shared taste by looking into genre or similar songs. And finally we’d incorporate any additional information such as mood (a common dictator in terms of selecting what to listen to) by considering attributes such as time of day or weather.



We put together 2 different versions of what this feature would potentially look like. 


Version 1

This version features a banner across the top of the screen that informs users that they can play music that they, Tom, and Julie would potentially enjoy.

We asked users what they thought would happen when they pressed the play button. 34 out of 44 users were successful. We calculated the completion rate and concluded that the probability the observed proportion 0.77 comes from a population greater than 0.7 is 85.11%. 

We also asked users how confident they are on their answer on a scale of 1-5 (1 being not at all confident, 5 being extremely confident). We can be 90% confident that the range is from 4.029 - 4.471.

TLDR: We can be fairly certain that at least 70% of users would be able to successfully understand the banner and users are very confident in their understanding of this feature.


Version 2

This version features playlists (similar to what Spotify already has) that would be potentially enjoyed by the user, Tom, and Julie. 

We asked users what they thought would happen when they pressed "Shared Mix 1". 7 out of 10 users were successful. We calculated the completion rate and concluded that the probability the observed proportion 0.7 comes from a population greater than 0.7 is 48.38%.

We also asked users how confident they are on their answer on a scale of 1-5 (1 being not at all confident, 5 being extremely confident). We can be 90% confident that the range is from 3.716 - 4.484. 

TLDR: We aren't at all certain that at least 70% of users would be able to successfully understand "Shared Mix 1" but users are fairly confident in their understanding of the feature.



Disclaimer: I did not work on this section. Engineering work was conducted by an engineer.

The sample project demonstrates the intended functionality of discovering nearby friends and finding matching music to play. Utilizing the Spotify and geolocation API a user is able to login and see matches of their top artists with their friends.

how information gets populated

Information gets populated in the fields through multiple steps: (1) The user authenticates with the Spotify API to allow artist information to be fetched, (2) authenticated user calls the Spotify API to get account info such as user name and queries top artists for the account, (3) the geolocation API returns coordinates of where the user is located, (4) the backend server checks for other friend user data in memory that has geolocation coordinates within a specific threshold and (5) displays artists that both friends have to allow people to play music they both like.

  1. Authorization

Uses the Spotify authentication API and sets application scope/permissions.

var state = generateRandomString(16);
res.cookie(stateKey, state);

// your application requests authorization
var scope = 'user-top-read user-read-private user-read-email';
res.redirect('' +
    response_type: 'code',
    client_id: client_id,
    scope: scope,
    redirect_uri: redirect_uri,
    state: state

2. Spotify API

Uses the Spotify v1 API to get user information and top artists from and

    url: '',
    headers: {
        'Authorization': 'Bearer ' + access_token
    success: function(response) { =;
    userProfilePlaceholder.innerHTML = userProfileTemplate(response);

    url: '',
    headers: {
        'Authorization': 'Bearer ' + access_token
    success: function(response) {
        var d = response;
        var topArtists = [];
        for(i=0; i<d.items.length; i++){
        userObject.topArtists = topArtists;

3. Geolocation API

Uses the built in geolocation API to get latitude and longitude.

navigator.geolocation.getCurrentPosition(function(position) {
    userObject.latitude = position.coords.latitude;
        userObject.longitude = position.coords.longitude;

4. Backend Server Check

Code assumes all demo user data is scoped as friends, searches through data given the distance threshold (i.e. users should be nearby) and returns matching top artists.

function getMatches(userId, lat, long){
    var matches = [];
    var topArtists = getTopArtists(userId);
    if(outsideDistanceThreshold(lat, long, userData[k])){      }
            var matchObj = { 'artist': singleTopArtist, 'friend': friendId};
return matches;

5. Display Match Results

Function tied to an event (i.e. on application load, navigation, etc) triggers the server to look for nearby friends and return matches.

function checkMatches(){
    var xhr = new XMLHttpRequest();
    xhr.onreadystatechange = function() {
        if (xhr.readyState == XMLHttpRequest.DONE) {
          data = xhr.responseText;
          var p = document.getElementById('content');
          p.innerHTML = data;
 = "";
    var url = 'matches?id=' + + '&lat=' + userObject.latitude + '&long=' + userObject.latitude;'GET', url, true);

next steps

Next steps include figuring out a way of seamlessly detecting nearby users in a way that doesn't feel intrusive and continuing to iterate based on user feedback.



Examine other detection candidates:

Geolocation API’s

Pros: Easy to implement, devices with GPS or native hardware often give accurate results

Cons: Have to implement or prompt user for initial permissions to have code run

Server Side Detection

Pros: Easy to implement, streaming services require device to connect to service

Cons: IP geolocation data is often non specific or inaccurate to have fine grain results

Device to Device Sensing

Pros: Novel way to detect nearby devices using ultrasonic or radio broadcast (ie bluetooth) and pass information

Cons: Difficult to implement properly, channels are often noisy and no standardized API across devices requiring additional engineering effort



Redefine match results:

Currently matches are just top artist names but could be expanded to more complex, possibly better techniques. The top API currently returns popularity scores for each artists which could result in a more nuanced version of the current demo. In addition, there are other top objects available such as genre which would allow composite and more cohesive union of users tastes by factoring in more data to make a better decision.



Implement feature on server:

Implementing code on actual Spotify services are different but similar to the external API, figure out requirements and tests needed to replicate a production feature.

The Challenge

Measuring the behavior of people interacting with interfaces is a critical part of developing products such as websites. There are two major types of usability tests: formative tests and summative tests. Summative tests can be described as “the usability of an application using metrics.” It can be further broken down into benchmark and comparative tests. The main goal of a benchmark test is to describe the usability of a product relative to a goal. It can also be used as a baseline for any future design changes. A comparative test involves the comparison between two different designs of the same product or the comparison of two competing products.

For this project I assessed the usability of the Mercedes Benz website by the means of one benchmark test and one comparative test.

Disclaimer: This project is not sponsored or affiliated with Mercedes Benz. It was done purely for academic purposes.


Benchmark Test:

If a user knows what car they want, they are going to want to figure out if there are any in-stock at their closest dealership. Users were presented the task of finding out how many Mercedes Benz AMG S65 Sedans there are at the dealership closest to the zip code of 91101. If users got the numerical value correct (which is 5), they are assigned the value of 1. If users inputted anything other than 5, they were assigned a 0. The goal of the test is to find out if at least 70% of users are able to complete the task. We are therefore testing the following hypotheses:

H0 : p = 0.7 versus H1 : p > 0.7

The best point estimate of p, the proportion of the population with a certain characteristic can be expressed by:

p̂ = x / n

For this task I utilized Amazon Mechanical Turk to obtain a random sample of 50 potential users and found that 30 out of 50 are able to successfully complete the task, so p̂ = 30/50. To determine if a sample proportion of 0.6 is statistically significant, we build a probability model. If we took a second random sample of 50 users, it will likely result in a different sample proportion. Since np (1 - p) = 50 (0.7) (1 - 0.7) = 10.5 10 and the sample size (n = 50) is sufficiently smaller than the population size (if we assume there are at least N = 1,000 users), we can use a normal model to describe the variability in p.̂ The mean of the distribution of p̂ is μp̂ = 0.7 since we assume the statement in the null hypothesis is true and the standard deviation of the distribution of p̂ is σp̂ = (p (1 - p) / n = 0.7 (1 - 0.7) / 50 0.065.

The level of significance is α = 0.05 so that we only have a 5% chance of making a Type I error. We want to know if it is unusual to obtain a sample proportion of 0.6 or more from a population whose proportion is assumed to be 0.7.

The test statistic is:

z0 =p̂-p0 /(p0 (1-p0)/n)
= (0.6 - 0.7) /
(0.7 (1 - 0.7) / 50) = -0.1 / 0.0648 = -1.54

Because this is a right-tailed test, we determine the critical value at the α = 0.05 level of significance to be z0.05 = 1.645. The test statistic z0 = -1.54 is less than the critical value of 1.645, we do not reject the null hypothesis.

There is not sufficient evidence at the α = 0.05 level of significance to conclude that more than 70% of users are able to successfully complete the assigned task.

It is up to individual companies to set a bar of their minimum task completion rate. For example, if Mercedes Benz wants at least 70% of all users to be able to successfully complete this task, they will need to continue to iterate and make changes to their website until the goal is met. Another way to look at it is to use this statistic as a benchmark for future website edits. 


Mercedes Benz



Comparative Test:

It can be useful to measure up your own website with competitors that are offering similar products. Using Amazon Mechanical Turk, I collected the time required to find safety features of 2 vehicles with similar price points (the Mercedes Benz C250 Coupe and the BMW 428i Coupe) from 34 random potential users on the Mercedes Benz website and the BMW website. The goal of the test is to confirm if the Mercedes Benz website is easier to navigate (as far as finding safety features are concerned). Half of the users were required to complete the task on the Mercedes Benz website first and the other half of the users were required to complete the task on the BMW website first.

This is a matched-pairs design because the variable is measured on the same set of users for both websites. We first compute the difference between the time taken on the Mercedes Benz website and the BMW website. If the time taken on the Mercedes Benz website is less than the BMW website, we can expect the Xi - Yi values to be negative. We are testing the hypotheses of:

H0:μd =0 versus H1:μd <0 with an α=0.05 level of significance.

The sample mean is d-bar = -35.3235 seconds and the sample standard deviation is sd = 79.1855 seconds.
The test statistic is:

t0 = d-bar0 / (sd / n) = -35.3235 / (79.1855 / 34) = -2.601

Because this is a left-tailed test, we determine the critical value at the α = 0.05 level of significance with 34 - 1 = 33 degrees of freedom to be -t0.05 = -1.692

The test statistic, t0 = -2.601 lies in the critical region, we reject the null hypothesis. There is sufficient evidence at the α = 0.05 level of significance to conclude that the time taken to find safety features on the Mercedes Benz C250 Coupe is less than the time taken to find safety features on the BMW 428i Coupe.

It is safe to say that it is quicker to navigate to and find safety features on the Mercedes Benz website. While speed is not everything it is a vital part of any product. We can either continue to test it against other similar websites to conclude that the process is sufficiently quick. When prioritizing requirements for website updates for any potential changes to the workflow of finding safety features, I would label it as low priority since it seems to be working well.