Designing and validating the methodology for the Internet assessment of fish consumption at a university setting

RESEARCH PAPER

Designing and validating the methodology for the Internet assessment of fish consumption at a university setting

Erika da Silva MacielI, ^*; Julia Santos Vasconcelos^II; Luciana Kimie Savay da Silva^III; Jaqueline Girnos Sonati^IV; Juliana Galvão^II; Dirceu da Silva^V; Marília Oetterer^II

^ICentro Universitário Luterano de Palmas - CEULP/ULBRA, Palmas, TO, Brasil, e-mail: [e-mail]

^IIDepartamento de Agroindústria, Alimentos e Nutrição, Escola Superior de Agricultura "Luiz de Queiroz" - ESALQ, Universidade de São Paulo - USP, Piracicaba, SP, Brasil

^IIIDepartamento de Alimentos e Nutrição, Faculdade de Nutrição, Universidade Federal de Mato Grosso - UFMT, Cuiabá, MT, Brasil

^IVDepartamento de Enfermagem, Faculdade de Ciências Médicas, Universidade Estadual de Campinas - UNICAMP, Campinas, SP, Brasil

^VFaculdade de Educação, Universidade Estadual de Campinas - UNICAMP, Campinas, SP, Brasil

ABSTRACT

Assessing fish consumption is complex and involves several factors; however, the use of questionnaires in surveys and the use of the Internet as tool to collect data have been considered promising approaches. Therefore, the objective of this research was to design a data collection technique using a questionnaire to assess fish consumption by making it available on a specific home page on the Internet. A bibliographical survey or review was carried out to identify the features of the instrument, and therefore pre-tests were conducted with previous instruments, followed by the Focus Group technique. Specialists then performed an analysis and conducted an online pre-test. Multivariate data analysis was applied using the SmartPLS software. The results indicate that 1.966 participants belonging to the University of São Paulo (USP) community participated in the test, and after the exclusion of some variables, a statistically significant results were obtained. The final constructs comprised consumption, quality, and general characteristics. The instrument consisted of behavioral statements in a 5-point Likert scale and multiple-choice questions. The Cronbach's alpha reliability coefficient was 0.66 for general characteristics, 0.98 for quality, and 0.91 for consumption, which indicate good reliability of the instrument. In conclusion, the results proved that the Internet assessment is efficient. The instrument of analysis allowed us to better understand the process of buying and consuming fish in the country, and it can be used as base for further research.

Keywords: fish consumption; structural equation modeling; Internet; questionnaire.

1 Introduction

In recent years, many studies have been conducted to assess the benefit of fish consumption to human health and in chronic diseases prevention. However, the results are inconsistent due to the difficulty to assess the consumption of fish and fish products (Sirot et al., 2011; Mina et al., 1007). Fish consumption is affected by several factors, namely the socioeconomic profile of consumers, their dietary standards, personal features, health conditions, and behavioral statements (Trondsen et al., 2003).

The need to assess dietary standards in distinct populations led to the use of questionnaires and, more recently, other methods such as the telephone and the Internet (Vanderslice et al., 2008). Questionnaires are used to collect information on behaviors, attitudes, opinions, and preferences (Rodrigues et al., 2005). The low cost and short time for filling out are the major advantages of using questionnaires. However, the number of studies on the validation of dietary intake is limited in Brazil (Ribeiro et al., 2009).

The use of the Internet and new technologies has allowed great changes in the educational systems, mainly at higher-learning institutions, which demonstrates how the use of high technology may allow the introduction of important innovative methods (Bargas-Avila et al., 2009).

The widespread use of computers in organizations has allowed new survey methods. In companies, the use of intranet has been increasing substantially, and it can be used as a tool to apply questionnaires (Stanton & Rogelberg, 2001; Bargas-Avila et al., 2009).

The use of the Internet to carry out surveys in institutions has brought many benefits, such as cost reduction, increased number of participants, and easiness to handle and analyze data (Stanton & Rogelberg, 2001). However, care should be taken when collecting data via the Internet, due to response variability, interruptions when filling out the forms, and lack of monitoring of the survey firm (Cronk & West, 2002). Collecting data online allows us to conduct a survey worldwide, mainly among populations that are difficult to access; on the other hand, it requires meticulous analysis to release the study and to select samples that ensure quality and validity of the results (Cantrell & Lupinacci, 2007). In face of the difficulty to assess fish consumption, this study aimed to design an online data collection instrument at a university community, as well as to validate its consistency using multivariate data analysis.

2 Materials and methods

The Focus Group survey technique was used; it consists of a survey conducted by a specialized moderator, and its application allows an in-depth analysis of the issue (Malhotra, 2008). This survey technique can also be characterized as a resource to understand the analytical process of perceptions, attitudes, and social representations of human groups (Morgan, 1997; Veiga & Gondim, 2001). The Focus Group technique may be used to address several issues, such as the understanding of perceptions, preferences, and consumer's behavior for a given product. Moreover, it can be used to analyze a product acceptability, to assess new products, and to collect information to develop surveys and generate quantifiable hypothesis (Malhotra, 2008).

First, an experienced specialist in conducting the Focus Group technique conducted the survey using semi-structured interviews in March 2009, at ESALQ - USP (College of Agriculture "Luiz de Queiroz" of the University of São Paulo - Brazil). Sixteen volunteers from ESALQ and UNICAMP (State University of Campinas - São Paulo - Brazil) were invited to participate in the Focus Group survey.

The instrument was then submitted for evaluation to personnel specialized in the survey techniques. The instrument was available and accessed through the Internet. Subsequently, it was submitted to a group of potential participants for its semantic validation, that is, to ensure that the scale items were understandable to the participants (Pasquali, 2003). Afterwards, the structural model to be tested was defined. Multiple-choice question items were constructed using a rating scale of attitude - 5 point Likert-type scale - next, they were grouped according to the main features: "Consumption", "Purchase", "Quality" and "Nutrition and Knowledge".

The samples comprised graduate and undergraduate students, faculty members, and staff of USP (University of São Paulo). USP has a population basis of 108,636 (Universidade de São Paulo, 2010). The non-probabilistic sampling was used for convenience, in which the participants were invited by e-mail to participate voluntarily in the survey. This sampling technique adopted is less costly, the sampling units are more accessible, and it is easy to measure. However, there are limitations in terms of population representativeness and care must be taken in generalizing results (Malhotra, 2008; Perez & Zwicker, 2010). The project was submitted to the Ethics Committee for Research (CEP) at ESALQ - USP and was approved under the protocol No.21 (COET/046).

The instrument was made available on a specific homepage and e-mails were sent to invite USP employees and students to voluntarily participate to collect research data. The survey was disseminated through social media, websites of some research institutes, electronic newsletters, speeches, brochures, and posters that were posted on bulletin boards all over campus. On the website, the participant had access to the Statement of Informed and Free Consent (IFC) of the survey and contact with the researchers, and, therefore, they could choose to fill out the forms by providing their institutional e-mail and USP ID number.

After a bibliographic review and the adoption of the Focus Group technique to construct the scales, a quantitative study was conducted. It allowed to empirically corroborate factors that influence fish consumption and to evaluate how these factors relate to the use of the appropriate statistical technique to validate the scale.

Subsequently, the relationship between the variables of interest was verified. To identify the relationship between the factors adopted in the modeling method, the Structural Equation modeling - SEM - was used. SEM is used to study complex phenomena that involve factors (latent variables) that cannot be directly measured (Pasquali, 2003; Universidade de São Paulo, 2010).This method includes several designated models such as the Covariance Analysis of Structure (CAS), the Latent Variable Analysis (LVA), the Confirmatory Factorial Analysis (CFA), and the Path Analysis (PA); it was also referred to as the LISREL analysis in the literature during the 1980s. SEM uses simultaneously a series of dependency relations, which is particularly useful when a dependent variable becomes independent in subsequent relations of dependency (Universidade de São Paulo, 2010).

The structural model relates to dependent and independent variables. The SEM models generally involve measurement and structural theories. The measurement theory specifies how the measured variables represent logically and systematically the constructs involved in a theoretical model, that is, the theory specifies a series of relationships that suggest how measured variables represent a latent construct that is not directly measured (Hair Jr. et al., 2009; Gefen et al., 2009; Hershberger et al., 2003). The exogenous (independent) and endogenous (dependent) variables are the constructs or latent variables of the study and are represented by ellipses. The variables or indicators are represented by rectangles, the arrows represent the casual relationships between the latent variables or the effect of variables in the measurement models (Maruayama, 1998).

The software programs SPSS 15.0 and SmartPLS - PM 2.0 M3 (Partial Least Square - Path Modeling) were used. The reliability test was performed to evaluate the threshold level for error-free, and therefore, it presents consistent results in the calculus of Cronbach's Alpha Coefficient (Rodrigues et al., 2005). A minimum criterion to assess the scales is the value of the Cronbach's Alpha Coefficient between 0.6 and 0.7, as long as the other indicators for the validation of the model conduct are good. High reliability of the constructo indicates the existence of internal consistency, which means that all measurements are consistent with the same construct (Hair Jr et al., 2009). The validation of the measurement models of each construct was performed through successive adjustments. The adjustment indexes were checked in each process.

3 Results

A total of 1,966 volunteers belonging to the USP community were considered for the data analysis. They were graduate and undergraduate students, faculty members, and staff of all USP campuses. The results show that special attention should be given to the interpretation of the results in the samples. The results indicate a dominant proportion of male, single, undergraduate students (Table 1). The average age was 30-42 years old (±12.41), and women were the majority (61.6%).

The first SEM analysis was carried out with all measurement indicators of latent variables with partial estimation in Partial Least Squares-Path Modeling (PLS-PM) (Chart 1).

The initial model tested (Figure 1) shows problems of factorial load related to the CARAC construct comprised of general characteristics of consumption and fish quality, including a negative load (CCS1), besides the other non-satisfactory indicators of adjustments (Table 2). The CS construct refers to consumption, the CN to nutrition and knowledge, and CQ to quality.

To assess the converging validity, we should check if the AVE is higher than 0.5 (greater than 50%), which meets the criterion recommended (Hair Jr et al., 2009). The index of the composite reliability analysis and that of the Cronbach's Alpha of each construct must be higher than or equal to 0.7, which evaluates if the indicator adequately measures the construct (Hair Jr et al., 2009; Zwicker et al., 2008).

The discriminant validation determines whether the variable is adequately discriminated. The test is carried out using the square root of the AVE, which must be higher than the correlation between the constructs (Zwicker et al., 2008). In the converging validity, the load of each indicator must be greater than 0.70, high in the latent variable, and low in the other variables (Zwicker et al., 2008). The Cronbach's Alpha Coefficient is also an assessment parameter, and the values higher than 0.5 are acceptable. However, the general recommendation includes values between 0.6 and 0.7; above this level, there is an indicative of good internal consistency (Hair Jr et al., 2009). Considering the criteria for the defined adjustments, there were some problems in the CARAC construct regarding the AVE, in which the values were lower than those recommended (0.3452) and those in the Cronbach's Alpha Coefficient, in which there was a lower value in the CN construct (knowledge and nutrition). This occurred because this construct has only three indicators (Hair Jr et al., 2009).

The adjustment process was applied, and the indicators that had lower factorial load were removed from the model individually, evidencing the low index of multiple square correlation (R²) (Zwicker et al., 2008). Therefore, considering all the established criteria, the initial model, which originally comprised 24 indicators, consisted of 15 indicators, as it was necessary to remove nine variables to make the adjustment (Chart 2). It was observed that of the nine variables removed only three belonged to the Likert scale, while the other variables belonged to the multiple choice questions.

After analyzing the data with the excluded variables, the result in the structural model (Figure 2) was obtained. All variables in the CQ, CARAC, and CS constructs showed factorial loads higher than 0.7 and most variables above 0.8, which indicates that there is convergent validity (Figure 2) (Chin 1998). The adjustment indexes in the SEM analysis were also satisfactory (Table 3).

These results show that with the exclusion of the problematic variables, there was an improvement in the AVE indicators, which remained above 0.5 and the R² increased. The reliability of the constructs was assessed in the Cronbach's Alpha Coefficient and in the response reliability, and all results (Table 3) remained above the minimum threshold recommended of 0.7 (Hair Jr et al., 2009). The indicators in the CARAC and CS constructs greatly improved with the elimination of the problematic variables.

4 Discussion

The initial model of measurements, containing all indicators, submitted to the SEM analysis, showed measurement adjustments that, in general, greatly differed from the levels considered reasonable considering the converging validity, discriminant validity, and latent variable reliability or constructs. The factorial loads attributed to these variables suggest that there is a theoretical problem regarding the way the questions were constructed and their respective response scales. For example, the item CCS1 "how often do you usually eat fish?" may be difficult for the participant to assess, which may lead to a possible mistaken response. Another problematic item is CCN1 "do you believe fish is beneficial to health?" because of some reports in the literature of fish causing diseases to consumers due to environmental contamination and inadequate handling during the fish processing, which may also lead the participant to confusion or incomprehension of the item.

This aspect should be considered since the main objective of the SEM is to assess the construct validity, that is, the level at which the items measured actually reflect the theoretical latent construct. Although the items CN1 and CN2 show limiting factorial loads when other adjustment criteria are used, such as in the AVE, the high level of loads shows variance problems in the construct. SEM does have a unique objective to establish isolated causal relation. For the interpretations in the SEM to be adequate, the idea of cause/effect should be abandoned and the idea of predictor/consequence should be adopted. This way, SEM can help clarify the causal relations that may not be implicit in correlated data (Mueller, 1997).

Considering the measurement features (nature and asymmetry), the estimation method of Partial Least Square (PLS) has structural advantages for the treatment of equation models. The PLS introduces the notion of latent or non-observable variable. It allows specifying the nature of relationships between latent variables and their indicators and the treatment of several explanatory variables in the same model. It also allows approaching problems of asymmetry in the variables and considering measurement errors (Falk & Miller, 1992). It is believed that the variables initially excluded do not correspond to the constructs and to the purpose of their measurements. This can result from the construction of the variable itself, and it is possible that the variables may have been poorly constructed and, therefore, did not fit the model.

The final model shows that assumptions in the literature, mainly related to quality and fish consumption, corroborate the need to include these constructs in the assessment instruments of the consumer's perception of fish consumption in Brazil. The main features related to consumption corroborate the items observed in the Focus Group analysis, based on the theory presented in several studies (Oetterer, 2002; Vinholis & Azevedo, 2002; Sonoda, 2006; Food and Agriculture Organization of the United Nations, 2009). On the other hand, it seems that for the sample studied, items such as frequency, amount and place of consumption, as well as the knowledge and importance of the fish nutritional value are not so expressive variables in the proposed model. The variables related to the product quality should be highlighted. This strong correlation of the variables related to product quality, consumption, and general characteristics seem to indicate a market trend in which food safety and quality have become vital for the purchase of the product (Federação das Industrias do Estado de São Paulo, 2010). The analysis of the final adjustment criteria shows that the measurement model is adequate for the validity and reliability of the constructs, as well as for the construction of the structural model presented.

However, the use of SEM requires a theoretical justification to specify the relations of dependence and modifications of the relations proposed (Hair Jr et al., 2009). Thus, it seems that explanatory studies that measure concepts do not allow to predict all variables due to the subjectivity involved. Accordingly, two main results may be expected from the SEM analysis. First, an estimation of the magnitude of the effects established between the variables is presented. These estimations are dependent on the fact that the specified model (diagram) is correct. Second, we can test if the model is consistent with the observed data, and if it is, we can say that it is plausible; however, it is not possible to affirm if it is correct (Maruayama, 1998; Klem, 2006).

In our study, the final model was not consistent with the initial model with respect to latent variables (constructs), and it was necessary to exclude the consumption and nutrition constructs and six variables of the CARAC, in which five of the variables were related to consumption and one with quality. The items in the CARAC construct were built with multiple-choice questions. The literature, however, shows that there was no obstacle for the SEM analysis with variables of different scales, which is supposedly one of the reasons for the low factorial loads observed.

The previous analysis of the literature for fish consumption is the main aspect for the adjustment of the SEM analysis. Based on the premise that the path analysis is related to models of one-direction causal flux, in which the measurements of each concept variable are perfectly reliable, it is believed that there is no measurement error (mensuration) or specification of the variables (structure). Each measurement is seen as an exact representation of the theoretical variable. One can expect that in these areas where human behavior has been investigated, such as in the case of the consumer behavior, assuming that there is such a thing a perfect reliability is unreal (Maruayama, 1998). SEM is more of a method of confirmatory analysis, more guided by the theory than empirical results. However, SEM offers the estimation for the load of all relations theoretically predicted (Hair Jr et al., 2009). The available data refer to the impact of a variable on another, as well as to the relationship of an indirect influence of a variable located between two others, in this case the CARAC construct. We must consider that the sample studied is composed, mostly, of college students and that changes occur in the life style during college life, such as the changes concerning the food diet quality and other behavioral changes that may affect the results (Soriano et al., 2000). For the dietary assessment, self-administered questionnaires or interviews are usually used. The use of information and communication technologies for dietary assessment has been recently reviewed and shown desirable results (Touvier et al., 2001).

The Internet facilitated data collection; on the other hand, interpretation errors, for example, cannot be controlled, requiring care with the specific terminology and with item formulations. It seems that even using a sample of a college community, there was misinterpretation of some items, which is not possible to assess. However, studies show that data collected using the Internet have more prediction validity and fewer systematic errors than data collected from interviews on the telephone. The use of the Internet may favor researchers in several fields (Chang & Krosnick, 2010). The period of one month has been approved as enough time to collect data online (Batagelj & Vehovar, 1998). The rate of unanswered questionnaires online is roughly half of the rate in a paper questionnaire (Hanscom et al., 2002). In our study, due to the difficulties to release, this period was extended to 90 days, when the number of accesses started to reduce and reached stability. Previous tests were carried out to investigate possible flaws in data transmission, which resulted in the precaution measures, such as the copy of personal data automatically sent to two specific e-mail addresses and the real time access to the information that was registered. We also considered the design of the website, the availability of questions, the facilities about specific terminologies, and the optimization of the time to fill out the form.

The Internet has several uses, such as communication, services, commerce, and marketing (Santos, 2001). The online methodology is a viable approach to conduct surveys due to its speed to collect data, low costs, and no geographical limitations (Cantrell & Lupinacci, 2007; Touvier et al., 2001). However, a study carried out with 1,186 college students in Italy aimed to understand the use of new technologies, using techniques based on quantitative and qualitative approaches, which included an administrative questionnaire applied through the Internet, showed that there is discrimination between the generations regarding the use of the Internet to search for information. Therefore, other actions should be taken when reaching, through online surveys, groups that have different positions regarding the Internet use (Ferri et al., 2009).

Therefore, the comparison between the conventional methodologies with the interview in paper and the online questionnaire shows that there are similarities in the data analyses and in the psychometric properties, which indicate the positive use of the Internet for this purpose (Ferri et al., 2009). Online surveys to collect data on dietary assessment, for instance, seem promising due to the positive cost-effective relation (Hanning et al., 2009; Boeckner et al., 2002). The online survey may not be as representative and efficient as another conventional method, mainly when the targeted population constantly uses the Internet (Bargas-Avila et al., 2009; Stanton & Rogelberg, 2001; Bliven et al., 2001). The online survey offers a number of advantages that include the possibility to communicate with groups geographically dispersed, the potential for the internationalization of the study, and the optimization of the process to tabulate data (Matthys et al., 2007). Therefore, the characteristics of data collection via the Internet make the survey technically, financially, and practically viable (Matthys et al., 2007). The widespread use of computers at universities, the access to the Internet, and its practical use in the university community are significant advantages to using this methodology for the population studied.

5 Conclusion

The results show that the process to construct and design the tool was essential to obtain the statistical results in the final analysis. The use of the Focus Group method allowed to adjust the instrument focusing on the consumer's perception for fish consumption, and the results are in line with those reported in the literature for the same matter.

Therefore, the analyses of the proposed instrument show good statistical indicators. The AVE indicator that reflects the general amount of variance in the indicators explained by the latent construct obtained values above 50% in all constructs for general characteristics 59%, quality 88%, and consumption 79%. The Cronbach's Alpha Coefficient also showed, after the removal of the problematic variables, values above the recommended, which indicated a good internal consistency, that is, the instrument measured the proposed objectives in the constructs, at least in the sample studied. Regarding the communality of the constructs, it was observed that after the exclusion of the variables that compose the construct nutrition, the results showed values above those recommended for the elimination (40%) in all constructs for general characteristics 59%, quality 88%, and consumption 79%.

It is concluded, therefore, that SEM was efficient in analyzing the causal relations between the constructs, which indicates concordance between the literature and the final model proposed in this research. Thus, the reproducibility of the instrument is possible in other samples with minor adjustments.

Acknowledgments

The authors are grateful for the financial support provided by FAPESP (São Paulo Research Foundation) and to CNPq (National Counsel of Scientific and Technological Development) for the scholarship provided.

Received 08 Feb., 2014

Accepted 03 Apr., 2014 (006280)

• Bargas-Avila, J. A., Lotscher, J., Orsini, S., & Opwis, K. (2009). Intranet satisfaction questionnaire: development and validation of a questionnaire to measure user satisfaction with the Intranet. Computers in Human Behavior 25(6) 1241-1250. http://dx.doi.org/10.1016/j.chb.2009.05.014
• Batagelj, Z., & Vehovar, V. (1998). Technical and methodological issues in surveys. In Proceedings of the 53º Annual Conference of the American Association for Public Opinion Research (AAPOR), St. Louis. Retrieved from http://www.ris.org/ris98/stlouis
• Bliven, B., Kaufman, S., & Spertus, J. A. (2001). Electronic collection of health-related quality of life data: validity, time, benefits and patient preference. Quality Life Research 10(1) 15-22. http://dx.doi.org/10.1023/A:1016740312904
• Boeckner, L., Pullen, S. C. H., Walker, S. N., Abbott, G. W., & Block, T. (2002). Use and realiability of Word Wide Web Version of the block health habits and history questionnaire with older rural women. Journal of Nutrition Education and Behavior 34, S20-S24. http://dx.doi.org/10.1016/S1499-4046(06)60307-2
• Cantrell, M. A., & Lupinacci, P. (2007). Methodological issues in online data collection. Journal of Advanced Nursing 60(5) 544-549. PMid:17973718. http://dx.doi.org/10.1111/j.1365-2648.2007.04448.x
• Chang, L., & Krosnick, J. A. (2010). Comparing oral interviewing with self-administered computerized questionnaires an experiment. Public Opinion Quarterly 74(1) 154-167. http://dx.doi.org/10.1093/poq/nfp090
• Chin, W. W. (1998). The partial least squares approach to structural equation modeling. In G. A. Marcoulides (Ed.), Modern methods for business research (pp 295-336). Mahwah: Lawrence Erlbaum Associates.
• Cronk, B. C., & West, J. L. (2002). Personality research on the Internet: a comparison of web based and traditional instruments in take-home and in-class settings. Behavior Research Methods Instruments and Computers 34(2) 177-180. http://dx.doi.org/10.3758/BF03195440
• Falk, R. F., & Miller, N. B. (1992). A primer for soft modeling New York: Akron Press.
• Federação das Industrias do Estado de São Paulo - FIESP. (2010). Brasil food trends 2020 São Paulo: FIESP. Retrieved from http://www.brasilfoodtrends.com.br/Brasil_Food_Trends/index.html
• Ferri, P., Cavalli, N., Costa, E., Mangiatordi, A., Mizzella, S., Pozzali, A., & Scenini, F. (2009). Italian university students and digital technologies: some results from a field research. Communications in Computer and Information Science 49(3) 357-365. http://dx.doi.org/10.1007/978-3-642-04757-2_38
• Food and Agriculture Organization of the United Nations - FAO. (2009). The state of world fisheries and aquaculture (SOFIA) Rome: FAO.
• Gefen, D., Straub, D. W. & Boudreau, M. C. (2009) Structural equation modeling and regression: guidelines for research practice. Communications of the Association for Information Systems 4(1) 1-77.
• Hair Jr., J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2009). Análise multivariada de dados (6 ed.). Porto Alegre: Bookman.
• Hanning, R. M. D., Royall, J. E., Toews, L., Blashill, L., Wegener, J., & Driezen, P. (2009). Web-based food behaviour questionnaire: validation with grades six to eight students. Canadian Journal of Dietetic Practice and Research 70 172-178. http://dx.doi.org/10.3148/70.4.2009.172
• Hanscom, B., Lurie, J. D., Homa, K., & Weinstein, J. (2002). Computerized questionnaires and the quality of survey data. Spine 27(6) 1797-1801. PMid:12195074. http://dx.doi.org/10.1097/00007632-200208150-00020
• Hershberger, S. L., Marcoulides, G. A., & Parramore, M. M. (2003). Structural equation modeling: an introduction. In B. H. Pugesek, A. Tomer & A. V. Eye (Eds.), Structural equation modeling: applications in ecological and evolutionary biology Cambridge: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511542138.002
• Klem, L. (2006). Structural equation modeling. In L. G. Grimm & P. R. Yarnold (Eds.), Reading and understanding more multivariate statistics (pp 227-260). Washington: American Psychological Association.
• Malhotra, N. (2008). Pesquisa de marketing: uma orientação aplicada (5 ed.). Porto Alegre: Bookman.
• Maruayama, G. M. (1998). Basic of structural equation modeling Thousand Oaks: Sage Publications.
• Matthys, C., Pynaert, I., De Keyzer, W., & Henauw, S. (2007). Validity and reproducibility of an adolescent Web-based food frequency questionnaire. Journal of the American Dietetic Association 107(4) 605-610. PMid:17383266. http://dx.doi.org/10.1016/j.jada.2007.01.005
• Mina, K., Fritschi, L., & Knuiman, M. (2007). A valid semiquantitative food frequency questionnaire to measure fish consumption. European Journal of Clinical Nutrition 61 1023-1031. PMid:17299496. http://dx.doi.org/10.1038/sj.ejcn.1602617
• Morgan, D. (1997). Focus group as qualitative research (pp 80-88). London: Sage Publications. Qualitative Research Methods Series 16.
• Mueller, R. O. (1997). Structural equation modeling: back to basics. Structural Equation modeling 4(4) 353-369. http://dx.doi.org/10.1080/10705519709540081
• Oetterer, M. (2002). Industrialização do pescado cultivado Guaíba: Agropecuária.
• Pasquali, L. (2003). Psicometria: teoria dos testes na psicologia e na educação Petrópolis: Vozes.
• Perez, G., & Zwicker, R. (2010). Fatores determinantes da adoção de sistemas de informação na área de saúde: um estudo sobre o prontuário médico eletrônico. Revista de Administração Mackenzie 11(1) 174-200. http://dx.doi.org/10.1590/S1678-69712010000100008
• Ribeiro, R. C. L., Panato, E., Silva, M. M. S., Rosado, L. E. F., & Rosado, G. P. (2009). Validade relativa de um questionário de freqüência alimentar para utilização em adultos. Revista de Nutrição 22(1) 81-95 http://dx.doi.org/10.1590/S1415-52732009000100008
• Rodrigues, D. S., Ramos, R. A. R., & Mendes, J. F. G. (2005). A system to evaluate and monitor quality of life in a University Campus. In Proceedings of the 9th International Conference on Computers in Urban Planning and Urban Management (CUPUM), London. Retrieved from http://repositorium.sdum.uminho.pt/bitstream/1822/5064/1/Rodrigues_CI_1_2005.pdf
• Santos, F. V. (2001). A importância da atividade física na qualidade de vida disseminada numa página da internet (Dissertação de mestrado). Universidade Federal de Santa Catarina, Florianópolis.
• Sirot, V., Dumas, C., Leblanc, J. C., & Margarit, I. (2011). Food and nutrient intakes of French frequent seafood consumers with regard to fish consumption recommendations: results from the CALIPSO study. British Journal of Nutrition 105(9) 1369-1380. PMid:21205369. http://dx.doi.org/10.1017/S0007114510005027
• Sonoda, D. Y. (2006). Demanda por pescado no Brasil entre 2002 e 2003 (Tese de doutorado). Universidade de São Paulo, Piracicaba.
• Soriano, J. M., Molto, J. C. & Manes, J. (2000). Dietary intake and food pattern among university students. Nutrition Research 20(9) 1249-1258. http://dx.doi.org/10.1016/S0271-5317(00)00217-7
• Stanton, J. M., & Rogelberg, S. G. (2001). Using Internet/intranet web pages to collect organizational research data. Organizational Research Methods 4(3) 200-217. http://dx.doi.org/10.1177/109442810143002
• Touvier, M., Kesse-Guyot, E., Méjean, C., Pollet, C., Malon, A., Castetbon, K., & Hercberg, S. (2001). Comparison between an interactive web-based self-administered 24 h dietary record and an interview by a dietitian for large-scale epidemiological studies. British Journal of Nutrition 105(7) 1055-1064. PMid:21080983. http://dx.doi.org/10.1017/S0007114510004617
• Trondsen, T., Scholderer, J., Lund, E., Lund, E., & Eggen, A. (2003). Perceived barriers to consumption of fish among Norwegian women. Appetite 41(3) 301-314. http://dx.doi.org/10.1016/S0195-6663(03)00108-9
• Universidade de São Paulo - USP. (2010). Anuário estatístico Brasil: USP. Retrieved from http://sistemas3.usp.br/anuario
• Vanderslice, J., Laflamme, D. M., & Wynkoop-Simmons, K. (2008). Comparing fish consumption information collected via a telephone survey and a food frequency questionnaire. Epidemiology 19(6), S315-S315.
• Veiga, L., & Gondim, S. M. G. (2001). A utilização de métodos qualitativos na ciência política e no marketing político. Opinião Pública 2(1) 1-15.
• Vinholis, M. M. B., & Azevedo, P. F. (2002) Segurança do alimento e rastreabilidade: o caso BSE. RAE Eletrônica 1(2) 1-19. http://dx.doi.org/10.1590/S1676-56482002000200008
• Zwicker, R., Souza, C. A., & Bido, D. S. (2008). Uma revisão do modelo do grau de informatização de empresas: novas propostas de estimação e modelagem usando PLS (Partial Least Squares). In Anais do 32º Encontro da ANPAD, Rio de Janeiro. CD-ROM.

*

Corresponding author

Publication Dates

• Publication in this collection
  05 Aug 2014
• Date of issue
  June 2014

History

• Received
  08 Feb 2014
• Accepted
  03 Apr 2014