EMPIRICAL COMPARISON OF THE MULTIDIMENSIONAL MODELS OF ITEM RESPONSE THEORY IN E-COMMERCE

Tezza, Rafael; Bornia, Antonio Cezar; Spenassato, Débora; Trierweiller, Andréa Cristina

doi:10.1590/0101-7438.2016.036.03.0503

person Rafael Tezza *

schoolCentro de Ciências da Administração e Socioeconômicas, Universidade do Estado de Santa Catarina, Av. Madre Benvenuta, 2037, 88035-001 Itacorubí, Florianópolis, SC, Brasil. E-mail: rafael.tezza@udesc.br

person Antonio Cezar Bornia

schoolDepartamento de Engenharia de Produção e Sistemas, Universidade Federal de Santa Catarina, Caixa Postal 476, Campus Universitário, 88010-970 Florianópolis, SC, Brasil. E-mail: cezar.bornia@ufsc.br

person Débora Spenassato

schoolInstituto de Matemática, Estatística e Física, Universidade Federal do Rio Grande, Campus Carreiros, Av. Itália Km 8, 96201-900 Rio Grande, RS, Brasil. E-mail: debospenassato@gmail.com

person Andréa Cristina Trierweiller

schoolDepartamento de Tecnologias da Informação e Comunicação, Universidade Federal de Santa Catarina, Rua Prefeito Walter Belinzoni, 219, 88900-017 Araranguá, SC, Brasil. E-mail: andreatri@gmail.com

person*Corresponding author.

Analysis	Target
Tetrachoric correlation matrix	Evaluation of the number of dimensions
Parallel analysis	Evaluation of the number of dimensions
Factor analysis method of restricted information	Evaluation of the number of dimensions and Analysis dimensional structure
Factor analysis method of full information	Evaluation of the number of dimensions and Analysis dimensional structure
Unidimensional model of two parameters IRT	Assessment of the adequacy of the unidimensional structure and multiple unidimensional structure
Multidimensional model of two parameters compensatory MIRT	Assessment of the adequacy of the multidimensional structure of IRT
Bifactorial model IRT	Assessment of the adequacy of the Bifactorial model IRT

Dimension	1	2	3	4	5	6	7	8	9	10
Eigenvalue	11.0	5.7	4.4	3.9	3.4	3.1	2.7	2.5	2.3	2.2
Accumulated proportion of explained variation	14.6	22.2	28.0	33.3	37.8	41.9	45.5	48.8	51.9	54.8

	χ²	Degrees of liberty	p-value	AIC	BIC
Mod1				23049.49	23648.21
Mod2	542.489	74	<0.001	22655.00	23549.09
Mod3	335.394	73	<0.001	22465.61	23651.07
Mod4	219.207	72	<0.001	22390.40	23863.25
Mod5	138.058	71	<0.001	22394.34	24150.59
Mod6	-1848.45	70	<0.005	24382.79	26418.44
Mod7	47.51	69	0.9775	24473.28	26784.34
Mod8	25.239	68	0.9999	24584.04	27166.52

Item	Parameters
	Discrimination	Standard Error	Difficulty	Standard Error
57	1.77	0.26	1.16	0.13
58	6.68	3.79	0.80	0.20
60	1.93	1.01	-2.64	0.67
69	0.64	0.18	-1.52	0.39
71	1.00	0.29	-2.10	0.46
74	8.91	58.30	0.58	0.83
75	0.84	0.18	-0.59	0.16

Item	a1	Se	a2	se	a3	se	a4	se	MDISC	d	se	MDIFF
03	0.30	1.03	1.52	3.50	0.07	39.49	-0.040	54.53	1.55	4.07	1.24	-2.62
05	-0.32	22.88	0.28	31.36	-0.40	34.43	-0.880	20.25	1.06	5.23	9.19	-4.95
06	1.09	1.70	1.91	15.07	0.30	54.56	-0.250	64.98	2.23	4.62	1.76	-2.07
08	-0.14	0.59	0.15	39.66	-0.41	35.75	-1.210	16.01	1.29	0.81	0.32	-0.63
10	-0.12	0.67	0.65	41.81	0.06	19.93	1.140	23.05	1.32	2.08	0.53	-1.58
12	-0.13	0.78	1.03	42.84	0.87	42.55	-0.680	23.18	1.52	2.49	0.58	-1.64
19	0.67	0.81	0.90	28.51	0.51	14.99	0.920	23.47	1.54	1.46	0.48	-0.95
21	1.43	1.46	0.89	38.57	1.89	15.48	0.970	30.04	2.71	5.12	2.41	-1.89
22	0.56	0.98	0.82	32.96	0.57	34.70	-0.590	19.83	1.29	3.75	1.02	-2.91
23	0.55	0.90	1.08	18.07	0.59	17.63	0.610	28.72	1.48	3.55	0.66	-2.40
24	11.44	177.69	5.62	257.53	16.62	283.21	7.090	291.11	22.11	46.63	719.38	-2.11
25	0.39	1.62	-0.11	99.18	2.82	24.08	-0.980	79.36	3.01	5.57	2.82	-1.85
27	0.06	0.56	0.62	40.23	-0.16	17.11	1.010	26.64	1.20	-0.60	0.34	0.50
28	0.15	0.72	0.04	52.30	0.42	41.15	1.560	10.25	1.62	-1.35	0.44	0.83
29	0.47	0.68	0.37	49.02	-0.36	23.97	1.150	22.52	1.34	-2.03	0.46	1.51
30	0.76	0.88	1.01	39.26	0.78	20.63	1.290	23.39	1.97	1.35	0.58	-0.69
32	0.62	0.65	0.65	18.39	0.65	10.41	0.610	14.00	1.27	1.58	0.42	-1.25
33	0.92	1.29	2.29	32.29	1.24	58.78	-0.010	61.27	2.76	5.55	2.87	-2.01
35	0.10	0.45	0.64	22.48	0.10	10.49	0.640	21.77	0.92	-0.35	0.30	0.38
37	0.62	0.77	1.45	11.39	-0.10	31.66	0.260	56.15	1.60	2.36	0.56	-1.47
38	0.95	0.70	0.47	33.23	0.70	21.06	1.100	10.75	1.68	0.67	0.48	-0.40
40	0.15	0.50	0.90	17.42	0.05	14.72	0.490	32.63	1.04	-0.15	0.28	0.14
43	0.01	0.86	1.06	9.27	-0.18	23.78	0.160	43.42	1.09	-4.20	1.18	3.86
45	0.28	0.66	1.13	25.77	0.05	17.52	0.710	40.90	1.36	1.49	0.41	-1.09
46	0.53	0.77	0.49	32.04	1.32	19.01	1.02	22.69	1.82	1.24	0.50	-0.68
47	0.60	3.15	1.07	48.33	1.16	41.31	-0.58	22.02	1.78	5.19	3.09	-2.91
48	0.90	1.04	1.62	20.43	0.79	28.94	0.67	44.95	2.12	3.06	0.80	-1.44
52	0.61	0.64	0.47	19.32	0.95	7.59	0.46	14.58	1.31	1.73	0.44	-1.32
55	0.45	0.92	0.23	41.62	1.16	16.83	-0.43	24.72	1.34	4.40	0.87	-3.29
56	1.91	4.34	1.81	18.87	0.93	41.42	0.33	49.98	2.81	6.88	16.99	-2.45
57	-2.08	0.72	-0.14	30.90	1.17	2.99	-0.03	35.93	2.39	-2.41	0.68	1.01
58	-50.47	398.55	8.56	309.51	6.92	181.90	10.11	217.53	52.64	-41.73	332.82	0.79
59	0.38	0.99	-1.19	27.54	1.34	46.79	0.69	76.93	1.96	3.80	1.45	-1.94
60	-2.74	4.63	-0.46	70.45	0.05	61.38	1.92	21.13	3.38	7.25	13.39	-2.15
61	-0.75	0.53	-0.40	21.09	0.95	25.62	0.62	38.81	1.42	0.38	0.28	-0.27
64	0.71	1.75	-1.92	38.71	1.75	56.92	0.33	115.42	2.71	6.39	3.54	-2.36
65	0.26	3.79	-2.53	56.83	2.89	84.56	0.98	160.92	3.97	8.67	20.84	-2.18
66	1.01	1.19	-0.14	27.72	0.74	29.10	0.94	24.34	1.57	4.87	2.89	-3.10
69	-0.86	0.56	0.32	12.34	-0.47	18.09	-0.40	23.44	1.11	1.11	0.37	-1.004
70	-0.35	0.44	-0.10	17.00	0.82	13.87	0.43	25.20	1.00	-0.57	0.27	0.573
71	-1.02	0.56	0.46	9.67	0.46	7.56	0.25	9.91	1.24	2.23	0.39	-1.81
73	0.82	11.08	-3.37	70.10	2.36	86.72	-0.22	185.21	4.20	10.71	37.53	-2.55
74	-4.25	1.69	0.60	25.59	1.23	10.37	0.62	19.79	4.51	-3.05	1.27	0.68
75	-0.79	0.37	0.29	15.64	0.33	13.21	-0.24	6.18	0.94	0.51	0.23	-0.54

Item	General	a1	a2	a3	a4	d	Item	General	a1	a2	a3	a4	d
3	0.531	0	0.374	0	0	2.116	43	0.277	0	2.097	0	0	-4.691
5	-0.483	0	0	0	-0.178	-0.178	45	0.627	0	0.274	0	0	0.793
6	0.755	0	0.676	0	0	2.342	46	0.999	0	0	0.584	0	0.69
8	-0.447	0	0	0	-0.839	0.538	47	0.672	0	0	0.402	0	2.591
10	0.522	0	0	0	0.747	1.293	48	1.089	0	0.722	0	0	1.79
12	0.106	0	0.831	0	0	1.418	52	0.709	0	0	0.253	0	0.956
19	0.948	0	0	0	0.021	0.832	55	0.388	0	0	0.741	0	2.519
21	1.804	0	0	1.104	0	3.364	56	1.31	0	0.321	0	0	3.426
22	0.233	0	0.73	0	0	2.2	57	-0.203	-1.064	0	0	0	-1.209
23	0.79	0	0.333	0	0	2.03	58	-0.513	-3.581	0	0	0	-2.966
24	6.115	0	0	4.418	0	12.924	59	0.243	0	0	0.9	0	1.925
25	0.385	0	0	0.997	0	2.252	60	-0.045	-1.175	0	0	0	2.986
27	0.482	0	0	0	0.151	-0.332	61	0.102	0	0	0.495	0	0.161
28	0.632	0	0	0	0.567	-0.768	64	0.12	0	0	1.796	0	3.903
29	0.505	0	0	0	0.218	-1.074	65	0.102	0	0	2.676	0	5.447
30	1.286	0	0	0	0.428	0.822	66	0.69	0	0	0	-0.155	2.572
32	0.805	0	0	0.068	0	0.916	69	-0.324	-0.387	0	0	0	0.616
33	1.146	0	1.127	0	0	3.182	70	0.228	0	0	0.411	0	-0.337
35	0.456	0	0.412	0	0	-0.233	71	0.051	-0.635	0	0	0	1.253
37	0.597	0	0.648	0	0	1.317	73	-0.103	0	-0.694	0	0	3.168
38	0.953	0	0	0	0.151	0.353	74	-0.253	-2.751	0	0	0	-1.829
40	0.519	0	0.076	0	0	-0.103	75	-0.091	-0.509	0	0	0	0.296

Model	Log of maximum verisimilitude	AIC	BIC	RMSEA	M2
Unidimensional Model	13826	14002.7	14362.6	0.07	2844.39
MIRT Model	12616	13016.1	13833.9	0.04	1405.22
Bifactorial model	13200	13464.1	14003.8	0.05	1937.95

Website	User orientation during navigation	Accessibility and reliability of the system	User control or user interaction with the system	Presentation of information
001	0,105	0,130	-0,864	0,677
002	0,367	0,524	-0,753	0,040
003	-0,887	-0,328	-0,557	0,525
004	0,782	0,324	-0,879	0,314

Item	Description of the item	Source
01	When opening the homepage do pop-up windows open?	^{Storey et al. (2002}); ^{Petre et al. (2006}); ^{Tezza et al. (2011})
02	Does the homepage present the site’s main content areas (navigation)?	^{Nielsen (2000})
03	Does the homepagemake clearwhat the site does (demonstrate the main products and or a brief description of its objective and or benefits that it offers), without needing the roll-bar?	^{Nielsen (2000})
04	Does the homepage present a summary of the most important sales?	^{Nielsen (2000})
05	Does the site have moving images that can distract the user?	^{Colesca (2007})
06	Do the links for sales go directly to the sale announced?	^{Nielsen (2000})
07	Are the menus in alphabetical order?	^{Olsina et al. (2001})
08	Does the site have a cascademenu?	^{Nielsen & Loranger (2006})
09	For navigation, is there a track on the left or links on top of the page?	^{Nielsen & Tahir (2002})
10	Are sub-categories grouped?	^{Nielsen & Tahir (2002})
11	Does the title of the window (browser) list the name of the site in the first place?	^{Nielsen & Tahir (2002})
12	Are there information for telephone contact or an address?	^{Nielsen (2000}); ^{Nielsen & Tahir (2002});
13	Do the page have a consistent visual appearance, that is do they always have the same visual appearance?	^{Aladwani & Palvia (2002}); ^{Ferreira & Nunes (2008})
14	Does the site have the option to access it in other languages?	^{Colesca (2007})
15	Do the clickable words (in color or underlined) have a distinct form when selected?	^{Nielsen & Loranger (2006})
16	Do the field labels begin with a capital letter, and are the remaining letters in lower case?	^{Aldwani & Palvia (2002})
17	Are the titles aligned to the left?	^{Inkpen et al. (2006})
18	Are text paragraphs separated?	^{Zerfass & Hartmann (2005})
19	Are apparently clickable words in fact clickable?	^{Nielsen (2000})
20	Are the title for screens, windows and dialog boxes on top, centered or left justified?	^{Inkpen et al. (2006})
21	Do all the pages have a search field?	^{Nielsen (2000})
22	When there is rolling, are there design elements (in the initial screen) thaat appear with end of page markers?	^{Nielsen & Loranger (2006})
23	Does the company logo in the upper left corner on all the site pages?	^{Roth et al. (2010})
24	Is there a link with a single click that leads to the homepage?	^{van der Merwe & Bekker (2003})
25	Does the site allow navigating its pages in only one window, that is, there is not opening of new windows in amid the navigation?	^{Nielsen & Loranger (2006}); ^{Tezza et al. (2011})
26	Are there different colors for previously visited links?	^{Nielsen & Tahir (2002})
27	Is there a list of frequently asked questions - FAQs?	^{Colesca (2007}); ^{Hernández et al. (2007}); ^{Tezza et al. (2011})
28	When entering search terms in the search field does the search engine offer suggestions?	^{Long & McMellon (2004})
29	Is the search system flexible in relation to the terms used by the user, that is, if the user types a term incorrectly, does the search system suggest a correction?	^{Agner (2012})
30	Do the search results allow classification by other criteria in addition to cost?	^{Nielsen (2000}); ^{Tezza et al. (2011})
31	Do long lists present indicators of communication, number of items and pages?	^{Nielsen & Loranger (2006}); ^{Tezza et al. (2011})
32	Are page continuation items visible?	^{Agner (2012})
33	Is the price of a product next to the image or link for the product?	^{Hung & McQueen (2004})
34	Is it possible to expand the photos of products to visualize details?	Novikova (2009); ^{Nielsen & Loranger (2006}); ^{Tezza et al. (2011})
35	In products in which there is more than one perspective, is it possible to visualize all the perspectives?	^{Nielsen & Tahir (2002}); ^{Tezza et al. (2011})
36	Are the groups of command buttons available in the column to the right, or on a line below the objects to which they are associated?	^{Inkpen et al. (2006})
37	Is there sufficient information about the products (size, basic characteristics)?	^{Signore (2005})
38	Is there a way for consumers to provide to insert feedback about the products?	^{Agner (2012})
39	Are there background images for the text?	^{Nielsen & Tahir (2002})
40	Does the site present related products at the end of the page?	^{Nielsen (2000})
41	Is there an option to share the page on social networks?	^{Agner (2012})
42	Is there an on-line help option?	^{Hernández et al. (2007})
43	Does the site have multimedia for product presentation?	^{Aladwani & Palvia (2002})
44	Does the company offer a free service, such as free shipping?	^{Nielsen & Tahir (2002})
45	Is there an indication that the site is safe at the time of making the purchase?	^{Nielsen & Tahir (2002})
46	When filling in the forms, can the user visualize the next steps in the interface?	^{Nielsen (2000})
47	Does the site have other payment forms in addition to a credit card?	^{Yoo & Donthu (2001}); ^{Hung & McQueen (2004})
48	Is it possible to know the total cost before registering (including shipping costs)?	^{Nielsen & Loranger (2006})
49	Are the fields used for the forms to be filled in by the user grouped linearly, avoiding unnecessary spaces?	^{Inkpen et al. (2006}); ^{Tezza et al. (2011})
50	When filling in a form, are options that are not valid or not available visibly deactivated to avoid errors?	^{Nielsen (2000})
51	When filling in a form, is information provided about how to complete the form?	^{Cybis (2007}); ^{Nielsen & Loranger (2006}); ^{Tezza et al. (2011})
52	Are required data differentiated from optional data in a visually clear manner?	^{Cybis (2007}); ^{Nielsen & Loranger (2006})
53	Is it possible to make a purchase without registering (which includes user name and password)?	^{Nielsen & Loranger (2006}); ^{Tezza et al. (2011})
54	Does the system provide audio signals when there is a problem with the data entered?	^{Nielsen & Loranger (2006}); ^{Tezza et al. (2011})
55	Are the error messages free of abbreviations and or codes generated by the operating system?	^{Cybis (2007})
56	Can any user action be taken back with the UNDO or BACK option?	^{Nielsen & Loranger (2006}); ^{Tezza et al. (2011})
57	Does all the non-textual content that is presented to the user have an alternative in text form that serves an equivalent purpose?	^{W3C (2008})
58	Can the information, structure and relations broadcast through the presentation be determined in a programmatic form or are they available in the text?	^{W3C (2008})
59	Is there another visual form of presenting information, beside color, to indicate an action, request a response or distinguish a visual element?	^{W3C (2008})
60	Does the visual presentation of text and images have a relation of contrast of at least 4.5:1?	^{W3C (2008})
61	Except for captions and text images, can the text be resized up to 200 percent without support technology, and without losing content or functionality?	^{W3C (2008})
62	Does the visual presentation of text and images have a contrast ratio of at least 7:1?	^{W3C (2008})
63	Are all the content functions operable through a keyboard interface without a need for any space of time between each individual item entered, except when the subjacent function requires entering data that depends on the sequence of actions by the user and not only on final points?	^{W3C (2008})
64	For each time limit defined by the content, is there control by the user?	^{W3C (2008})
65	For information in movement, in an intermittent mode, in shifting or automatic updating, is there an option for user control?	^{W3C (2008})
66	Is a mechanism available to ignore blockages of content that are repeated on various Web pages?	^{W3C (2008})
67	Do the Web pages have titles that describe the topic or finality?	^{W3C (2008})
68	Can the finality of each link be determined based only on the text of the link or based on the text of the link together with the respective context of the determined link in a programmatic form, except when the finality of the link is ambiguous for users in general?	^{W3C (2008})
69	Do the headers and the tags describe the topic or the finality?	^{W3C (2008})
70	Can the pre-defined human language for each Web page be determined in a programmatic manner?	^{W3C (2008})
71	Does changing the definition of a component of the user interface automatically provoke a change of context, at least when the user has been warned about this situation before using the component?	^{W3C (2008})
72	Are the components that have the same functionality in a set of Web pages identified in a consistent manner?	^{W3C (2008})
73	If an input error is automatically detected, is the item that has the error identified and is the error described to the user in text?	^{W3C (2008})
74	Are labels or instructions provided when the content requires inputting data by the part of the user?	^{W3C (2008})
75	In the content implemented using languages for marking, do the elements have complete marks at the beginning and end, are the elements fit according to the respective specifications, do the elements have duplicated attributes, and are all the IDs exclusive?	^{W3C (2008})