Filled once by every participant, after the 3rd block of the experiment.
The survey is ased on the SUS survey.
from utils import utils, sus
from utils import data as data_module
%matplotlib inline
usability_data = sus.load_table()
usability_data = sus.fillna(usability_data)
a, b = data_module.to_groups(usability_data)
sus.per_question_plot(a, b)
data_module.save_to_graphics('usability-per_question_statistics')
Calculate the one-sided t-test between the groups scores.
See note about convertion from two-sided t-test to one-sided here.
scores = usability_data.apply(sus.score, axis=1)
a, b = data_module.to_groups(scores)
utils.independent_one_sided_ttest_summary(a, b, ylabel='SUS score', ylim=(0, 100))
data_module.save_to_graphics('usability-sus_scores')
mean (a vs b): 45.227 67.292 stderr (a vs b): 6.859 2.997 t: -2.812328306916324 p: 0.007039373095872296 df: 21
Comparing the results to a known dataset. Based on Aaron Bangor , Philip T. Kortum & James T. Miller (2008) An Empirical Evaluation of the System Usability Scale, International Journal of Human-Computer Interaction, 24:6, 574-594.
means = a.mean(), b.mean()
studies_dataset = data_module.sus_dataset.multisurvey_studies
for group, mean in zip(['A', 'B'], means):
dataset_std = (mean - studies_dataset['mean']) / studies_dataset['std']
print('Group {} std from dataset: {}'.format(group, dataset_std))
Group A std from dataset: -2.060886880600444 Group B std from dataset: -0.2020499859589997