ABSTRACT
This paper aims to analyze the activity of two Brazilian tourism agencies in social media and the online behavior of their consumers. The research used Natural language processing resources supported by sentiment and content analysis techniques. The main results show a prevalence of positive comments on the companies' pages, and companies are more responsive to users on pages with a higher number of positive comments. There is also a tendency towards more significant company interaction with user comments that express positive emotions.
Keywords:
Content marketing; Social media; User-generated content; Natural language processing; Sentiment analysis
RESUMO
Este trabalho buscou analisar a atividade de duas agências brasileiras de turismo nas mídias sociais e o comportamento online de seus consumidores. A pesquisa utilizou recursos de processamento de linguagem natural, apoiados pelas técnicas de análise de sentimento e análise de conteúdo. Os principais resultados indicaram que há prevalência de comentários positivos nas páginas das empresas, sendo que as empresas são mais responsivas aos usuários em páginas que possuem um maior número de comentários positivos de seus clientes. Apontou-se, ainda, uma tendência de maior interação por parte das empresas com comentários de usuários que expressam sentimentos positivos.
Palavras-chave:
Marketing de conteúdo; Mídias sociais; Conteúdo gerado pelo usuário; Processamento de linguagem natural; Análise de sentimento
1. Introduction
When discussing marketing in the current times, it is indispensable to mention the influence and impacts of social media on many market segmentations, for its rise and mass adoption have been providing excellent business opportunities for companies in the marketing field. Evidence of this tendency is the increased relevance of digital marketing in that area of research and testing since, in addition to using extraordinarily efficient and low-cost tools for advertising campaigns, social media has become an important communication channel between consumers and companies in recent years (EVANGELISTA; PADILHA, 2014EVANGELISTA, T.; PADILHA, T. Monitoramento de postagens sobre empresas de e-commerce em redes sociais utilizando análise de sentimentos. In.: BRAZILIAN WORKSHOP ON SOCIAL NETWORK ANALYSIS AND MINING (BraSNAM), 2., 2014, Porto Alegre. Proceedings… Porto Alegre: SBCOPENLIB, 2014.).
In this context, the feedback generated by the comments can represent an influencing factor for consumers’ behavior and their purchasing decisions. The study “The 2012 Traveler” (THINK…, 2012), carried out by Google and the Ipsos MediaCT institute, reveals that when people are willing to plan their trip, approximately 37% of leisure travelers use the comment sections as sources, looking for reviews and opinions. According to Ye, Law, and Gu (2009YE, Q.; LAW, R.; GU, B. The Impact of Online User Reviews on Hotel Room Sales. International Journal of Hospitality Management, [s.l.], v. 28, n. 1, p. 180-182, mar. 2009.), online reviews written by hotel consumers have an important impact on hotel room sales. Therefore managers should seriously consider those reviews, especially the ones posted on third-party websites.
On the other hand, the way companies react to comments can also be an essential factor. As Pantelidis (2010PANTELIDIS, I. Electronic meal experience: a content analysis of online restaurant comments. Cornell Hospitality Quarterly, [s.l.], v. 51, n. 4, p. 483-491, ago. 2010.) describes, restaurant managers who respond appropriately to comments on electronic forums can turn dissatisfied consumers into loyal customers.
In general, the content of comments in company posts on social media presents unstructured information that, when processed through ratings, can be used as a support tool for performance indicators. According to Miranda and Sassi (2014MIRANDA, M. D.; SASSI, R. J. Using sentiment analysis to assess customer satisfaction in an online job search company. Business Information Systems Workshops. [s.l.], v. 183, [s.n.], p. 17-27, 2014.), sentiment analysis can be used as a support tool to enrich the assessment of consumer satisfaction. According to Stich, Emonts-Holley, and Senderek (2015STICH, V.; EMONTS-HOLLEY, R.; SENDEREK, R. Social media analytics in customer service: a literature overview - an overview of literature and metrics regarding social media analysis in customer service. In.: 11TH INTERNATIONAL CONFERENCE ON WEB INFORMATION SYSTEMS AND TECHNOLOGIES, 11., 2015, [s.l.], Proceedings… [s.l.]: SCITEPRESS - Science and Technology Publications, 2015.), researchers can use the technique to assess the "consumer experience."
In addition to sentiment analysis, natural language processing techniques can provide information from the content of consumer comments that leads to the detection of defects and opportunities for improvements in the product or service in question (MOGHADDAM; ESTHER, 2010MOGHADDAM, S.; ESTER, M. Opinion digger: an unsupervised opinion miner from unstructured product reviews. In.: PROCEEDINGS OF THE 19TH ACM INTERNATIONAL CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, 19., 2010, Toronto. Proceedings… Toronto: ACM Digital Library, 2010, p. 1825-1828.).
The growth of social media like Instagram and Facebook, which encourage the sharing of photos by their users, can represent an opportunity for sectors such as tourism, which can represent one of the activities that most use images to promote themselves and to attract consumers, primarily through sharing photos and experiences on social media (SANTOS et al., 2017SANTOS, G. C. O. et al. As redes sociais e o turismo: uma análise do compartilhamento no Instagram do Festival Cultura e Gastronomia de Tiradentes. Revista Iberoamericana de Turismo (RITUR), Penedo, v. 7, n. 2, p. 60-85, 2017).
Given the arguments and studies presented, as well as the growth of techniques that enable the analysis of a large amount of data, the volume of academic studies on the content of comments in posts by Brazilian companies on social media is relatively low (e. g. AMARAL et al., 2017AMARAL, F. et al. Comentários no TripAdvisor: do que falam os turistas?. Dos Algarves: a multidisciplinary e-journal, [s.l.], v. 2, n. 26, p. 47-67, 2017.; COELHO; GOSLING, 2015COELHO, M. F.; GOSLING, M. S. Comentar bem ou mal na internet? O engajamento de viajantes em reviews de hotéis. Revista: Turydes Revista Turismo y Desarrollo2015.; EVANGELISTA; PADILHA, 2014EVANGELISTA, T.; PADILHA, T. Monitoramento de postagens sobre empresas de e-commerce em redes sociais utilizando análise de sentimentos. In.: BRAZILIAN WORKSHOP ON SOCIAL NETWORK ANALYSIS AND MINING (BraSNAM), 2., 2014, Porto Alegre. Proceedings… Porto Alegre: SBCOPENLIB, 2014.). Also, most studies observe the need to use the available tools for sentiment analysis or other natural language processing techniques. Therefore, this article aims to analyze the social media activity of Brazilian consumers and companies in the tourism sector.
The information resulting from this type of research can contribute not only to more in-depth studies on the behavior of consumers and companies on social media but also to companies that want to increase their presence on social networks, seeking to leverage the visibility of their brand.
2 Theoretical reference
2.1 Content marketing
According to Baltes (2015BALTES, L. Content marketing-the fundamental tool of digital marketing. Bulletin of the transilvania university of brasov. Series V: economic sciences, Romania, v. 8, n. 2, p. 111-118, 2015.), content marketing has become the key to marketing campaigns' success and the essential tool in digital marketing. Visual resources, such as images, videos, and others, are often used to create content for social media. Vicente (2016VICENTE, M. Uma imagem vale mais que mil palavras? Humans Of New York e a febre de páginas que contam histórias de anônimos através de fotografias no Facebook. Revista Mangaio Acadêmico, [s.l.], v. 1, n. 1, p. 01-11, 2016.) points out that posts with images have advantages in promoting engagement over other types of posts. In addition, the author highlights the power of storytelling and narratives in the photo captions to significantly impact the consumer.
According to Forouzandeh, Soltanpanah and Sheikhahmadi (2014FOROUZANDEH, S.; SOLTANPANAH, H.; SHEIKHAHMADI, A. Content marketing through data mining on Facebook social network. Webology, [s.l.], [s.n.], n. 1, p. 1-11, 2014.), content marketing has a significant advantage compared to other types of marketing on social media. Users are not directly and suggestively introduced to this type of marketing. Instead, they are involved in another matter and are provided with helpful information they do not consider commercial, making them trust the content provider. Establishing a feeling of trust promotes users' loyalty. Lima (2014LIMA, V. M. Engajamento do consumidor em uma comunidade virtual de marca. 2014. 103 f. Dissertação (Mestrado em Gestão Empresarial) - Escola Brasileira de Administração Pública e de Empresas, Fundação Getulio Vargas, Rio de Janeiro, 2014.) considers this loyalty a consequence of an engagement with the brand. Therefore, content marketing and consumer engagement efforts are directly related.
The social media platform Facebook is an adequate basis for content marketing, through which different types of content can be easily presented and quickly distributed among users. Also, marketing professionals can combine Content marketing with other forms of marketing. By developing and promoting knowledge and information among users, content marketing creates a feeling of trust toward purchasing goods due to the content presented. In addition, this increase in trust among users expands the sale of goods. Content marketing is the knowledge to absorb customers indirectly. The results of the study by Forouzandeh, Soltanpanah and Sheikhahmadi (2014FOROUZANDEH, S.; SOLTANPANAH, H.; SHEIKHAHMADI, A. Content marketing through data mining on Facebook social network. Webology, [s.l.], [s.n.], n. 1, p. 1-11, 2014.) confirm such benefits of content marketing, as it can present various goods to users.
2.2 Social media in the tourism sector
For Zeng and Gerritsen (2014ZENG, B.; GERRITSEN, R. What do we know about social media in tourism? A review. Tourism Management Perspectives, [s.l.], v. 10, [s.n.], p. 27-36, Apr. 2014.), social media plays an increasingly important role in several aspects of tourism, especially regarding the search for information and decision-making behaviors (FOTIS; BUHALIS; ROSSIDES, 2012FOTIS, J.; BUHALIS, D.; ROSSIDES, N. Social media use and impact during the holiday travel planning process. In.: Fuchs, M.; Ricci, F.: Cantoni, L. Information and Communication Technologies in Tourism. Sweden: Spring, 2012. Available in: https://link.springer.com/chapter/10.1007/978-3-7091-1142-0_2. Access in: 15 mar. 2019.
https://link.springer.com/chapter/10.100...
), tourism promotion (BRADBURY, 2011BRADBURY, K. The growing role of social media in tourism marketing. In.: BRADBURY, K. Blog·bur·y. [s.l.]: [s.n.], 2011. Avaliable in: https://kelseybradbury.weebly.com/uploads/1/0/9/2/10927387/tourismsocialmedia-comm427.pdf. Access in: 07 jun. 2019.
https://kelseybradbury.weebly.com/upload...
) and focusing on best practices for interacting with consumers through social media channels (e. g. social sharing of vacation experiences). Many countries regard social media as essential to promote their tourism industries.
Leung et al. (2013LEUNG, D. et al. Social media in tourism and hospitality: a literature review. Journal of Travel & Tourism Marketing, [s.l.], v. 30, n. 1-2, p. 3-22, jan. 2013.) point out that several academics noted the ability of social media to help tourism and hospitality companies to engage potential customers, increase their online presence and thus lead to higher online revenues. For Agnihotri et al. (2016AGNIHOTRI, R. et al. Social media: Influencing customer satisfaction in B2B sales. Industrial Marketing Management, [s.l.], v. 53, [s.n.], p. 172-180, 2016.), the responsiveness acquired from informative communication with customers on social media positively correlates with consumer satisfaction. Gallaugher and Ransbotham (2010GALLAUGHER, J.; RANSBOTHAM, S. Social media and customer dialog management at starbucks. MIS Quarterly Executive, [s.l.], v. 9, n. 4, 2010.) highlight that social media makes it easier for companies to have a firm-to-consumer dialogue, strengthening consumer-to-firm and firm-to-consumer communication.
The authors also concluded that consumers generally used social media during the research phase of their travel planning process. In addition, reliability is a critical antecedent in determining their decision about using the information on social media. Finally, the article discusses the applications of social media in five main functions (promotion, product distribution, communication, management, and research). Based on the research results, social media is an important strategic tool in managing tourism and hospitality - particularly in promotion, business management, and research functions.
2.3 Sentiment analysis
The sentiment analysis technique seeks to create structured knowledge that a support system or decision-maker can use (ARAÚJO; BENEVENUTO; RIBEIRO, 2013ARAÚJO, M.; GONÇALVES, P.; BENEVENUTO, F. Measuring sentiments in online social networks. In.: PROCEEDINGS OF THE 19TH BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB, 19., 2013, Salvador. Proceedings… Salvador: ACM Press, 2013. p. 97-104.). Specifically, in marketing and Customer Relationship Management (CRM), sentiment analysis aims to detect favorable or unfavorable opinions about products and services by using a large amount of virtual data in text format collected from social media, such as social networks, website recommendations, forums, blogs, and other sources (MORENO, 2015MORENO, A. C. Análise de sentimentos na classificação de comentários online aplicando técnicas de text mining. 2015, 72 f. Dissertação (Mestrado em Sistemas Integrados de Apoio à Decisão) - Departamento de Ciências e Tecnologias de Informação, Instituto Universitário de Lisboa, Lisboa, 2015.).
Sentiment Analysis usually classifies emotions into two categories: positive versus negative, or into three categories, considering neutral comments and a polarity score (SHARDA et al., 2014) or even an opinion score (PANG; LEE, 2008PANG, B.; LEE, L. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, [s.l.], v. 2, n. 1-2, p. 1-135, 2008.).
According to Araújo, Benevenuto, and Ribeiro (2013ARAÚJO, M.; GONÇALVES, P.; BENEVENUTO, F. Measuring sentiments in online social networks. In.: PROCEEDINGS OF THE 19TH BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB, 19., 2013, Salvador. Proceedings… Salvador: ACM Press, 2013. p. 97-104.), there are currently two main approaches to sentiment analysis of textual productions. The first one, the supervised approach, is based on machine learning concepts, which start from defining characteristics that allow one to distinguish between sentences with different emotions by training a model with previously labeled sentences. The model then can identify the emotion in previously unknown sentences. The second approach, the unsupervised one, does not rely on machine learning model training and, in general, is based on lexical methods for treating emotions that involve calculating the polarity of a text based on the semantic orientation of the words it contains. Although previously labeled data is not necessary to carry out the training, its efficiency is directly related to the generalization of the vocabulary used, considering the various existing contexts.
Several sentiment analysis methods, supervised and unsupervised, are available in the literature. Ribeiro et al. (2016RIBEIRO, F. N. et al. Sentibench - a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science, [s.l.], v. 5, n. 23, 2016.) compare the predictive capacity of 18 different methods. To perform the comparison, The use of datasets containing texts that were earlier manually labeled with emotions and separated into two or three categories was necessary. The metrics used are accuracy, precision, recall, and the F1 score (GONÇALVES et al., 2013GONÇALVES, P. et al. Comparing and combining sentiment analysis methods. In.: PROCEEDINGS OF THE FIRST ACM CONFERENCE ON ONLINE SOCIAL NETWORKS, 1., 2013, [s.l.]. Proceedings… [s.l.]: ACM Press, 2013.). The authors point out that, although the results have identified some methods considered among the best for different datasets, the overall prediction performance still left much room for improvement. More importantly, the predictive performance of the methods varies widely across data sets. Consequently, the level of agreement between the methods greatly varies when analyzing the same body of text. As expected, the methods' two-class prediction capacity is considerably greater than their three-class prediction capacity.
In addition to the standard methods used and documented in the literature, specific methods are available, such as IBM Watson and Microsoft Text Analytics, which are implementations mainly focused on commercial applications. However, they have resources available for testing, through which their users are free to use tools with certain limitations (AGUIAR et al., 2018Aguiar, E. et al. Análise de Sentimento em Redes Sociais para a Língua Portuguesa Utilizando Algoritmos de Classificação. In.: Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, 36., 2018, Porto Alegre. Proceedings… Porto Alegre: [s.n.], 2018. p. 393-406,).
As Araújo et al. (2016) explain, most of those resources are available only in English, considering that this language dominates the content provided. However, some efforts have been present to develop emotion techniques in other languages, although there is little knowledge about the performance and the absolute need or feasibility of developing these solutions. Therefore, the authors propose using specific state-of-the-art methods for analyzing emotions in nine languages. To that end, they use data previously labeled in each language and a simple automatic translation into English and develop a methodology to compare and validate the results. However, there is still little research in the literature regarding the metrics of accuracy and precision of other existing methods, such as Watson Natural Language Understanding (NLU), in detecting the emotions in texts written in the Portuguese language (e. g., AGUIAR et al., 2018Aguiar, E. et al. Análise de Sentimento em Redes Sociais para a Língua Portuguesa Utilizando Algoritmos de Classificação. In.: Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, 36., 2018, Porto Alegre. Proceedings… Porto Alegre: [s.n.], 2018. p. 393-406,).
According to Gonzales and Lima (2003GONZALES, M.; LIMA, V. Recuperação de informação e processamento da linguagem natural. In.: Congresso da sociedade Brasileira de computação, 23., 2003, [s.l.]. Proceedings… [s.l.]: [s.n.], 2003. p. 347-395, p. 3)
Natural language processing (NLP) computationally handles the different aspects of human communication, such as sounds, words, sentences, and speeches, considering formats and references, structures and meanings, contexts and uses. In a vast sense, NLP aims to make the computer communicate in human language, not always necessarily at all levels of understanding and generation of sounds, words, sentences, and speeches.
According to Pang and Lee (2008PANG, B.; LEE, L. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, [s.l.], v. 2, n. 1-2, p. 1-135, 2008.), companies have paid more attention to collecting all information published about their products, services, reputation, and customers. In addition, consumers are also interested in knowing relevant information about products they may consume or what is being said about them (THINK…, 2012; YE; LAW; GU, 2009YE, Q.; LAW, R.; GU, B. The Impact of Online User Reviews on Hotel Room Sales. International Journal of Hospitality Management, [s.l.], v. 28, n. 1, p. 180-182, mar. 2009.). These facts point to a constant implementation by companies seeking information that provides competitive advantages of techniques brought by advances in information technology, especially in the digital environment.
2.4 Sentiment analysis in tourism
According to Moreno (2015MORENO, A. C. Análise de sentimentos na classificação de comentários online aplicando técnicas de text mining. 2015, 72 f. Dissertação (Mestrado em Sistemas Integrados de Apoio à Decisão) - Departamento de Ciências e Tecnologias de Informação, Instituto Universitário de Lisboa, Lisboa, 2015.), there needs to be more support in the literature concerning applying sentiment analysis techniques in tourism. Table 1 presents the literary studies carried out over the years related to applying sentiment analysis techniques in the context of the tourism sector.
The first publications made in the area appeared in 2007. They address the unexpected influence of social media on companies and the tourism industry (ZENG; GERRITSEN, 2014ZENG, B.; GERRITSEN, R. What do we know about social media in tourism? A review. Tourism Management Perspectives, [s.l.], v. 10, [s.n.], p. 27-36, Apr. 2014.) and the imminent loss of control over the information available on the network regarding both companies and the industry (DWIVEDI et al., 2007DWIVEDI, M.; SHIBU, T.; VENKATESH, U. Social software practices on the Internet: Implications for the hotel industry. International journal of contemporary hospitality management, [s.l.], v. 19, n. 5, p. 415-426, 2007.). With emphasis, Thevenot (2007THEVENOT, G. Blogging as a social media. Tourism and hospitality research, [s.l.], v. 7, n. 3-4, p. 287-289, 2007.) already indicated that if companies would not properly monitor online comments, the industry would have to deal with the consequences since blogs also generate negative impacts (THEVENOT, 2007THEVENOT, G. Blogging as a social media. Tourism and hospitality research, [s.l.], v. 7, n. 3-4, p. 287-289, 2007.).
3. Methodological procedures
Table 2 presents the research strategy. We can classify this research as exploratory. Malhotra (2001MALHOTRA, N. Pesquisa de marketing: uma orientação aplicada. 3. ed. Porto Alegre: Bookman, 2001., p. 106) defines exploratory research as "a type of research whose main objective is to provide criteria on the problem situation faced by the researcher and their understanding". In addition, the research used text mining techniques, classified by Hearst (1999HEARST, M. Untangling Text Data Mining. In.: PROCEEDINGS OF THE 37TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS ON COMPUTATIONAL LINGUISTICS, 37., 1999, College Park. Proceedings… College Park: University of Maryland, 1999. p. 3-10.) as exploratory data analysis. On the other hand, the data analysis process considered mainly quantitative factors, such as the rate of positive/negative comments. However, the analysis also benefited from the qualitative aspects of the data present in the content of the texts. Thus, the research followed a qualitative and quantitative approach.
3.1 Data collection and structuring
We carried out this research by collecting unstructured data. The sources for collection are the data contained in posts of two Brazilian travel agencies with great relevance in the market (LUCAS, c2013LUCAS, A. S. Top 10 maiores agencias de viagens do Brasil. [s.l.]: Top10+, c2013. (Luxo). Available in: https://top10mais.org/top-10-maiores-agencias-de-viagens-brasil/. Access in: 15 mar. 2019.
https://top10mais.org/top-10-maiores-age...
). The choice of companies was due to their online presence. Travel agencies with large numbers of followers on social media were selected to provide a greater volume of unstructured data for the survey. There is no delimitation on socio-demographic factors in the selection of post comments.
3.2 Data Analysis
3.2.1 Polarity of comments
To analyze the emotions expressed in the comments present in the posts, the iFeel tool, provided by the Department of Computer Science at the Federal University of Minas Gerais (ARAÚJO et al., 2014ARAÚJO, M. et al. iFeel: a system that compares and combines sentiment analysis methods. In.: THE 23RD INTERNATIONAL CONFERENCE ON WORLD WIDE WEB, 23., 2014, Seoul. Proceedings… Seoul: ACM Press, 2014. p. 75-78.), was primarily used. The tool, available in 60 languages, performs the sentiment analysis of texts and files using 18 methods available in the literature to achieve greater coverage and accuracy. First, the text is translated into English using the Yandex Application Programming Interface (API) to reach this mark and then analyzed. According to Reis et al. (2015REIS, J. et al. Uma abordagem multilıngue para análise de sentimentos. In.: BRAZILIAN WORKSHOP ON SOCIAL NETWORK ANALYSIS AND MINING (BraSNAM), 4., 2015, Porto Alegre. Proceedings… Porto Alegre: SBCOPENLIB, 2015.), this approach proved feasible since the translation of databases into other languages does not significantly interfere with the accuracy or the range of methods for sentiment analysis.
According to Araújo et al. (2016), the most accurate method of detecting emotions using machine translation is Sentistrength. This method uses a lexicon dictionary labeled by humans and enhanced by machine learning. However, methods like Umigon and Vader also performed well in different databases. In addition, according to Ribeiro et al. (2016RIBEIRO, F. N. et al. Sentibench - a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science, [s.l.], v. 5, n. 23, 2016.), the Sentistrength method tends to classify many sentences as neutral, which can decrease its accuracy in analyses considering three classes (positive, negative, and neutral). Thus, this research considers the results of the Sentistrength and Umigon methods. The results for each comment were reintroduced in the columns next to it in the spreadsheets initially set up.
The comments were also analyzed to complement the research and make comparisons using a script programmed in Python language, which uses the sentiment analysis feature available in the IBM Watson API. (Watson Natural Language Understanding), due to its ability to perform analysis natively in Portuguese (SOUSA, 2017SOUSA, A. R. Processamento automático de línguas naturais: Um estudo sobre a localização do IBM Watson™ para o português do Brasil. 2017. 76 f. Monografia (Bacharel em Línguas Estrangeiras Aplicadas ao Multilinguismo e à Sociedade da Informação) - Instituto de Letras, Universidade de Brasília, Brasília (DF), 2017.). In addition, there is a limitation in the iFeel tool that prevents the analysis of sentences with more than 300 characters. Therefore, comments that overcame this barrier were also analyzed separately through IBM Watson, employing a script that goes through the spreadsheets containing the iFeel results, identifies the comments with more than 300 characters, analyzes them, and saves the results. We carried out a test of the method's ability to identify the emotions present in the texts, according to the metrics used by Gonçalves et al. (2013GONÇALVES, P. et al. Comparing and combining sentiment analysis methods. In.: PROCEEDINGS OF THE FIRST ACM CONFERENCE ON ONLINE SOCIAL NETWORKS, 1., 2013, [s.l.]. Proceedings… [s.l.]: ACM Press, 2013.), to check the feasibility of using IBM Watson in its current state of development. For this, a database of comments written in Portuguese on social media Twitter was used, manually labeled by the MiningBR group (AGUIAR et al., 2018Aguiar, E. et al. Análise de Sentimento em Redes Sociais para a Língua Portuguesa Utilizando Algoritmos de Classificação. In.: Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, 36., 2018, Porto Alegre. Proceedings… Porto Alegre: [s.n.], 2018. p. 393-406,). The analysis results are at the end of the discussion and results section.
A script developed in Python that runs through each line of the spreadsheet counted and calculated the metrics. Due to its structure, we can associate comments with the respective post. We consider all comments except those posted on the official companies' websites. The analysis metrics chosen were: the ratio between positive and negative comments (P / NG), the ratio between positive and total (P / Total), and between negative and total (NG / Total). Since the measures of accuracy and precision of the methods present in the literature are higher in the analysis of two classes (positive and negative), we also present the metrics omitting the number of neutral comments, the ratio of each class by the sum of the two classes ( P / (P + NG) and N / (P + NG)).
By observing the metrics, we analyzed the differences between the results of the two social media of each company. A T-Student statistical test ensured the results' significance, comparing the recorded polarity averages with a 95% confidence interval and assuming different variances between the samples. For this test, we transform the results into values ranging from -1 (negative) to 1 (positive). We used the Levene test (SCHULTZ, 1985SCHULTZ, B. B. Levene's test for relative variation. Systematic Zoology, [s.l.], v. 34, n. 4, p. 449-456, dec. 1985.) to decide on the assumed equality or variance difference between samples. In the case of company A, since the observed Facebook comments are far older than the first Instagram posts, the T-Student test was carried out on a sample where all comments on both media are from between 09/09/2018 and 03/25/2019, under the premise that possible changes in the company's services or events over time can impact the average polarity of comments.
3.2.2 Companies responsiveness
An automated search in each spreadsheet analyzed the companies' responsiveness through the development of a script in the Python language (whose operation description is present in the Appendix) using the libraries: xlutils, xlrd, and xlwt and, to extract the entities present in the text, the SpaCy library for Natural Language Processing, which allows a process of separating the sentences into their respective grammatical and syntactic classes, in addition to extracting the entities present in the text. In 2015, a survey by Emory University and Yahoo! Labs showed that spaCy offered the fastest syntactic parser in the world and that its accuracy was among the best available (CHOI; TETREAULT; STENT, 2015CHOI, J.; TETREAULT, J.; STENT, A. It depends: dependency parser comparison using a web-based evaluation tool. In.: PROCEEDINGS OF THE 53RD ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, 58.; THE 7TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING, 7., 2015, Beijing. Proceedings… Beijing: ACL Anthology, 2015. p. 387-396. (Volume 1: Long Papers).).
In this way, each comment answered by the company, even those present in the responses of another comment, is associated with a company response. Finally, this newly created spreadsheet was scrolled down to search, based on the total of answered comments, the same metrics presented for the polarity of the total comments.
3.2.3 Aspects of improvement
The search for improvement aspects in the companies' service started with extracting nouns and their related adjectives and keywords in the text content, using the SpaCy library and the IBM Watson API NLU module available in Python.
The developed script ran through an aggregated spreadsheet containing comments on the pages of the two social media of each company, extracting a list of nouns from each comment and, if available, a list containing the associated adjectives. Those elements are inserted in a second spreadsheet so that each line has a noun and an adjective. After that process, the script goes through the second spreadsheet, counting the incidence of nouns and noun-adjective pairs. Thus, it was possible to create a table with the spreadsheet data containing the nouns of the highest incidence and their recurrently associated adjectives in order of highest incidence. Nouns whose meanings are too broad or do not contain the aspects of improvement have been disregarded (e. g., "time," "day," and the name of the companies). In order to search for non-identified relevant keywords with the search for nouns and adjectives, the keyword identification feature of Watson NLU scrolled through the spreadsheets containing the results of this tool's sentiment analysis.
To complement the insights generated from the most relevant nouns, an analysis of the individual content of a portion of the comments considered negative was made, according to the procedures proposed by Bardin (2011BARDIN, L. Análise de conteúdo. Lisboa: Edições, 2011. v. 70. (Edição revista e actualizada)).
In the pre-analysis stage, we defined that the comments would be analyzed assuming users reported some aspects of the service negatively. The objective was to find them. First, a spreadsheet for each company was separated, containing user comments classified as negative by at least one of the methods. The choice of documents followed the rule of representativeness. Next, we chose a portion of the negative comments for analysis. The previously identified nouns served as a guide in choosing comments so that, from the 15 most recurring nouns, were separate those with more specific meanings and related to aspects of service in the universe of tourism. From these words, we separate the five with the highest incidence.
From that, we defined using Excel to search for comments containing these words. The following script carried out the number of searches: In words or groups of words that totaled more than 200 incidences, it performed 100 searches on the spreadsheet for comments containing the word; in words whose incidence was less than 100, it performed 50 searches; when the incidence was less than 50, it searches all comments containing the word in question. Then, it classifies the comments into more minor reports such as "Ticket canceled, route or date changed," "Problem in the post-sale," or "undue charge." It then counts the incidence of these reports. The results of the inferences from these data are available in section 4.3.
4. Results and discussion
When observing the companies' posts, a pattern is identified, mainly in company A, where resources such as images, videos, and informative descriptions of places and experiences are used, as well as inspiring texts and narratives, in order to awaken the consumers' desire (VICENTE, 2016VICENTE, M. Uma imagem vale mais que mil palavras? Humans Of New York e a febre de páginas que contam histórias de anônimos através de fotografias no Facebook. Revista Mangaio Acadêmico, [s.l.], v. 1, n. 1, p. 01-11, 2016.) and to get them to seek the company's services. Thus, companies use content marketing strategies since users are not introduced directly and suggestively to the company's services, as Forouzandeh, Soltanpanah and Sheikhahmadi (2014FOROUZANDEH, S.; SOLTANPANAH, H.; SHEIKHAHMADI, A. Content marketing through data mining on Facebook social network. Webology, [s.l.], [s.n.], n. 1, p. 1-11, 2014.) explain. However, on the other hand, company B also uses many posts that resemble traditional advertisements, containing promotions and exposing its services.
4.1 Sentiment polarity
Table 3 illustrates the size of the samples in which the polarity analysis was performed, in addition to data such as the total of posts and the percentage of company comments concerning the total. It is important to emphasize that, for this stage, the research does not consider the comments of the analyzed companies.
Concerning company A, the discrepancy observed in the average amount of comments per post between the two media indicates that the posts on the Instagram page achieve greater success in promoting engagement with their customers. Thus, this platform may improve performance in promoting digital marketing (ARAÚJO, 2015ARAÚJO, R. Marketing científico digital e métricas alternativas para periódicos: da visibilidade ao engajamento. Perspectivas em Ciência da Informação, Belo Horizonte, v. 20, n. 3, p. 67-84, 2015.). Furthermore, the fact may be more likely when considering the differences in followers of the two media, since the company's Facebook page, at the time of the survey, had more than ten times the number of followers on the Instagram page (12,550,820 and 954,000 followers, respectively). However, regarding company B, the opposite situation is observed. Therefore, one possible hypothesis is that the posting style of company A, which uses more resources such as images and narratives in the caption, is more successful in promoting engagement on the social media Instagram than on Facebook, even though it has the network used as a basis by Vicente (2016VICENTE, M. Uma imagem vale mais que mil palavras? Humans Of New York e a febre de páginas que contam histórias de anônimos através de fotografias no Facebook. Revista Mangaio Acadêmico, [s.l.], v. 1, n. 1, p. 01-11, 2016.) to point out the advantages of this type of posting. However, it is important to note that such findings can also simply reflect that companies have different strategies and ways of investing on each social media.
As shown in Tables 4 and 5, positive comments are prevalent over negative ones. As expected, there is a large discrepancy between the results of the methods used, as reported by Ribeiro et al. (2016RIBEIRO, F. N. et al. Sentibench - a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science, [s.l.], v. 5, n. 23, 2016.). This fact, along with the high incidence of comments classified as negative, indicates plenty of room for improvement in the prediction of methods.
Both on Instagram and Facebook, positive comments are prevalent over negative ones. These results align with the results of Coelho and Gosling's (2015COELHO, M. F.; GOSLING, M. S. Comentar bem ou mal na internet? O engajamento de viajantes em reviews de hotéis. Revista: Turydes Revista Turismo y Desarrollo2015.) work in the context of hotel reviews, which shows a tendency for those who demonstrate engagement in social media regarding a product/service to have a positive perception of it. In addition, generally, there is no apparent difference in the proportions of negative and positive comments between the two media. However, there is a slightly higher incidence of negative comments on the Facebook page, which may indicate a greater tendency for users of this media to express negative feelings in their comments. To test the significance of this difference, the results of the T-Student test of the polarity mean are available (Table 6), considering the same period and assuming different variances, as well as the result of the Levene test that supports the choice of the line of analysis. In this case, we consider hypotheses H0 (there is no difference between the posts on the Facebook and Instagram pages of company A regarding sentiment) and H1 (there is a difference between the posts on the Facebook and Instagram pages of company A regarding sentiment).
Since the value of P (T <= t) is less than 0.05, and t Stat exceeds the value t Critical, we reject H0 and can affirm that there are differences in the average sentiment polarity on the posts. This finding may reinforce the hypothesis that Instagram has a more significant potential for success in obtaining the benefits that Vicente (2016VICENTE, M. Uma imagem vale mais que mil palavras? Humans Of New York e a febre de páginas que contam histórias de anônimos através de fotografias no Facebook. Revista Mangaio Acadêmico, [s.l.], v. 1, n. 1, p. 01-11, 2016.) pointed out of company A's posting style.
As seen in Tables 7 and 8, there are higher rates of positive comments with the comments on the company A page. In contrast to the previous case, on the Company B pages, there is a higher incidence of negative comments on the Instagram page over the Facebook page. This finding contributes in favor of formulating hypotheses contrary to the one in which users of Instagram media have a greater tendency to comment positively.
The result of the T-test of the comparison of means is available in Table 9. Again, we consider the H0 hypotheses (there is no difference between company B's page posts on Facebook and Instagram regarding sentiment) and H1 (there is a difference between the posts on company B's Facebook and Instagram pages regarding sentiment).
Since the P (T <= t) value is less than 0.05, and t Stat exceeds the critical t value, we reject H0. Therefore, we can affirm differences in the average sentiment polarity of the posts. However, the contradictory findings between the two companies make it difficult to conclude the presence of an apparent pattern in the tendencies of manifestations of emotions in the two different social media.
The apparent prevalence of positive comments over negative comments on the pages of the two companies on different social media may indicate a high level of consumer satisfaction with the provided service, as shown by the work of Miranda and Sassi (2014MIRANDA, M. D.; SASSI, R. J. Using sentiment analysis to assess customer satisfaction in an online job search company. Business Information Systems Workshops. [s.l.], v. 183, [s.n.], p. 17-27, 2014.). It is important to note that the tendency of the SentiStrength method to classify texts as a neutral class in a three-class analysis reported by Ribeiro et al. (2016RIBEIRO, F. N. et al. Sentibench - a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science, [s.l.], v. 5, n. 23, 2016.) is not observed in this research when compared with the Umigon method - which received, on average, the best performance measures for this type of analysis. However, we can observe that, in practically all the analyzed databases, more comments were classified as neutral by the Umigon method.
4.2 Companies’ responsiveness
Table 10 shows the general data on the number of companies' responses to customer comments. In both media, there is a low rate of response to comments. The large volume of comments on social media can increase the cost or even make it impossible to show attention to all or even most of the comments, despite the benefits in engagement and responsiveness brought by the firm-consumer dialogue pointed out by Gallaugher and Ransbotham (2010GALLAUGHER, J.; RANSBOTHAM, S. Social media and customer dialog management at starbucks. MIS Quarterly Executive, [s.l.], v. 9, n. 4, 2010.) and the positive impacts on consumer satisfaction pointed out by Agnihotri et al. (2016AGNIHOTRI, R. et al. Social media: Influencing customer satisfaction in B2B sales. Industrial Marketing Management, [s.l.], v. 53, [s.n.], p. 172-180, 2016.).
The differences in the percentages of answered comments show that companies have given different attention to the comments of each media since company A responds more to Instagram comments (Tables 11 and 12). In contrast, company B does the opposite (Tables 13 and 14). The media that receive the most attention from companies are the ones where the highest positive comments are present, respectively. This fact may indicate a possible correlation between a company's responsiveness in online communication and the polarity of emotions expressed in comments related to the company, and consequently in consumer satisfaction (MIRANDA; SASSI, 2014MIRANDA, M. D.; SASSI, R. J. Using sentiment analysis to assess customer satisfaction in an online job search company. Business Information Systems Workshops. [s.l.], v. 183, [s.n.], p. 17-27, 2014.), which is in agreement with the work of Agnihotri et al. (2016AGNIHOTRI, R. et al. Social media: Influencing customer satisfaction in B2B sales. Industrial Marketing Management, [s.l.], v. 53, [s.n.], p. 172-180, 2016.).
We observe that the proportion of positive comments answered to the total answered is more significant than that of the total positive with the total sample. Thus, the Instagram page's administrators tend to interact with comments that express positive emotions. This trend is valid from the view of digital marketing, knowing the benefits of positive comments in promoting the company (EVANGELISTA; PADILHA, 2014EVANGELISTA, T.; PADILHA, T. Monitoramento de postagens sobre empresas de e-commerce em redes sociais utilizando análise de sentimentos. In.: BRAZILIAN WORKSHOP ON SOCIAL NETWORK ANALYSIS AND MINING (BraSNAM), 2., 2014, Porto Alegre. Proceedings… Porto Alegre: SBCOPENLIB, 2014.). Thus, companies can engage more with people who comment positively to encourage them to continue speaking out about the company, promoting word-of-mouth marketing, as Gallaugher and Ransbotham (2010GALLAUGHER, J.; RANSBOTHAM, S. Social media and customer dialog management at starbucks. MIS Quarterly Executive, [s.l.], v. 9, n. 4, 2010.) pointed out.
This discrepancy is apparent with greater intensity on Facebook posts, in which the proportion of total positive comments to total comments is 24.57%. The same proportion for the sample of comments answered is 50%, approximately twice the total ratio. Compared to the table of emotions' polarity, we observe that, even with a higher rate of responses, the moderators of company B's page show a slight tendency, although less intense, to respond more to comments classified as positive over negative ones.
The discrepancy between the proportions of positive to the total answered comments could also be observed less intensely in company B. Therefore, in addition to the advantages explained by Gallaugher and Ransbotham (2010GALLAUGHER, J.; RANSBOTHAM, S. Social media and customer dialog management at starbucks. MIS Quarterly Executive, [s.l.], v. 9, n. 4, 2010.) and considering the findings of the research by Coelho and Gosling (2015COELHO, M. F.; GOSLING, M. S. Comentar bem ou mal na internet? O engajamento de viajantes em reviews de hotéis. Revista: Turydes Revista Turismo y Desarrollo2015.), it is possible to state that responding to positive comments is a way of maintaining a good relationship with users with a greater potential for engagement, a fact that can be identified in this research, when, in addition to the identified preference by positive comments, it is also apparent that the media that receive the most comments from the company are the ones that have the most positive comments from their customers.
In that way, companies engage customers who tend to comment positively on their social media pages. However, Gallaugher and Ransbotham (2010GALLAUGHER, J.; RANSBOTHAM, S. Social media and customer dialog management at starbucks. MIS Quarterly Executive, [s.l.], v. 9, n. 4, 2010.) and Evangelista and Padilha (2014EVANGELISTA, T.; PADILHA, T. Monitoramento de postagens sobre empresas de e-commerce em redes sociais utilizando análise de sentimentos. In.: BRAZILIAN WORKSHOP ON SOCIAL NETWORK ANALYSIS AND MINING (BraSNAM), 2., 2014, Porto Alegre. Proceedings… Porto Alegre: SBCOPENLIB, 2014.) also point to the need to monitor and respond to negative comments, given their high power to damage the company's image.
4.3 Aspects of improvement
The results indicate that, when manifesting some dissatisfaction on the page of company A, users sought to associate the company figure with various types of negative adjectives, as shown in Table 15. Words such as "service," "problem," "company," and "client," despite having expansive meanings in this context, present a north for further analysis since the high incidence of these words and their associated adjectives indicate that a considerable part of negative comments is related to problems in the company's services, especially in customer service, prompting the question of which aspects of that service would be presenting problems. The presence of negative comments regarding the company represents aspects of consumer-consumer or consumer-to-business communication (GALLAUGHER; RANSBOTHAM, 2010GALLAUGHER, J.; RANSBOTHAM, S. Social media and customer dialog management at starbucks. MIS Quarterly Executive, [s.l.], v. 9, n. 4, 2010.) that can negatively impact consumer confidence in the company, an element considered by Forouzandeh, Soltanpanah and Sheikhahmadi (2014FOROUZANDEH, S.; SOLTANPANAH, H.; SHEIKHAHMADI, A. Content marketing through data mining on Facebook social network. Webology, [s.l.], [s.n.], n. 1, p. 1-11, 2014.) as the biggest advantage of content marketing efforts.
Words like "ticket," "website," "trip," "money," and "package," among others and their associated adjectives, indicate more specific service aspects that have problems - for example, possible problems with the ticket. Also, the ticket price is a commonly mentioned element. Through the individual analysis of the comments, it was possible to identify the recurrence of the situation reported by Coelho and Gosling (2015COELHO, M. F.; GOSLING, M. S. Comentar bem ou mal na internet? O engajamento de viajantes em reviews de hotéis. Revista: Turydes Revista Turismo y Desarrollo2015.) of the presence, to a lesser extent, of travelers who engage in reporting complaints and negative occurrences in their experience.
The content of 100 reviews of the company's purchase and after-sales was analyzed. Of this sample, 29 people presented problems regarding the ticket not being sent by e-mail until the moment of the comment or problems in the ticket issuance on the website. It is important to note that most of these cases are related to fraud, reported by company A and customers. The fraudster impersonates the company, making a fake sale through the Facebook chat system. Thus, reports of what happened could be observed primarily in Facebook comments. The company made it clear when responding to comments that they do not make sales through Facebook - a necessary movement, as explained by Gallaugher and Ransbotham (2010GALLAUGHER, J.; RANSBOTHAM, S. Social media and customer dialog management at starbucks. MIS Quarterly Executive, [s.l.], v. 9, n. 4, 2010.), in correcting mistakes and mitigating damages. However, the analysis reveals misunderstandings in which people associated the company with fraud, potentially damaging the company's image (EVANGELISTA; PADILHA, 2014EVANGELISTA, T.; PADILHA, T. Monitoramento de postagens sobre empresas de e-commerce em redes sociais utilizando análise de sentimentos. In.: BRAZILIAN WORKSHOP ON SOCIAL NETWORK ANALYSIS AND MINING (BraSNAM), 2., 2014, Porto Alegre. Proceedings… Porto Alegre: SBCOPENLIB, 2014.). In addition, 21 people in the sample had problems where the airline company canceled the ticket or the route or date was changed, causing various inconveniences to customers.
It is important to point out that in some cases, customers reported a relation to the Avianca airline's recent financial and operational problems (BRANCO; CAVALCANTI; DOCA, 2018) and the company's difficulties in accomplishing the rerouting of flights in order to satisfy its entire clientele. Thirteen people reported not getting a refund, which, in most of the reports, was related to changes in the ticket's date carried out by the responsible company. Eleven people expressed their opinion that the ticket was expensive, eight people had difficulties purchasing the ticket, either on the website or in the app, and the same number had difficulties canceling or rescheduling. Four people showed dissatisfaction with the fee charged for rescheduling the trip. We can also observe in a smaller quantity: problems with undue collection, tickets without the seat's number, and other problems in the after-sales. Thus, Moghaddam and Esther (2010MOGHADDAM, S.; ESTER, M. Opinion digger: an unsupervised opinion miner from unstructured product reviews. In.: PROCEEDINGS OF THE 19TH ACM INTERNATIONAL CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, 19., 2010, Toronto. Proceedings… Toronto: ACM Digital Library, 2010, p. 1825-1828.) reported that natural language processing could help extract opportunities for improvement in those services. Table 16 exemplifies comments that report the observed problems.
Through the analysis of 50 incidences of the word "site" in negative comments, it is possible to observe that a large part of the comments also contained the word "ticket" in order to indicate that previously presented problems, such as difficulties in the purchase, cancellation or rescheduling, possibly have some relation to the functioning of the website, since customers purchase the tickets and packages through the company's website or official app. Additionally, reports of instability on the site or blocked customer access are present with a very low incidence.
The analysis of negative comments containing the words "trip," "money," or "package" (50 recurrences of each word) also indicates, in general, the presence of the same problems identified in the previous analysis, which states that in the comments containing the word "trip," there is a greater incidence in reports of problems such as changes in the flight and the trip's route or date. In addition, comments with the word "money" highlight problems with cancellations and refunds.
It is interesting to point out that from this sample, 19 comments are from people requesting the resolution of a specific case in which a woman who is a pastor and singer did not obtain her refund, which exemplifies the power of individuals who stand out due to their influence in certain groups to move people to the manifestation of causes, even though punctual ones, and impact companies' digital image, both negatively and positively, following the research by Silva and Tessarolo (2016SILVA, C.; TESSAROLO, F. Influenciadores digitais e as redes sociais enquanto plataformas de mídia. In.: CONGRESSO BRASILEIRO DE CIÊNCIAS DA COMUNICAÇÃO, 39., 2016, São Paulo. Proceedings… São Paulo: INTERCOM, 2016.).
Analogously to the case of company A, there is a high incidence of nouns such as "company," "problem," and "service" in company B's negative comments (Table 17), which indicates possible problems in the services provided by it. In addition, the high incidence of words such as "trip," "package," "ticket," "agency," "ship," "cruise," and "hotel," among others, guide the search for a content analysis of comments containing these words.
An analysis of 50 negative comments containing the word "trip" indicates that, in general, part of the problems presented above are identified for this company.16 of the comments indicated a problem in which the company canceled the trip or flight and rescheduled, causing complications for customers, including problems in reversing the amount paid. In most of those cases, a relation between the issues and the company Avianca was reported, a fact already mentioned previously.
It is important to note that seven people desire to travel using the company's services but cannot due to financial conditions or other reasons. From this sample, there are three comments in which customers report their intention to hire the company's services, but "fear" manifested after the negative comments. These reports exemplify the results of the study "The 2012 traveler" (THINK…, 2012), which states that, when planning their trips, certain people seek criticism and opinions in online comments, in addition to representing signs of a possible loss of confidence in the company's services. In addition, other punctual complaints are present regarding the price of the trip and the fees for rescheduling the date.
The analysis of 50 incidences of the word “package” exposed the problem with the company Avianca, representing a large part of the complaints, and identified four complaints concerning the lack of package options, specifically for the following destinations: the city of Fortaleza, the continent of Africa, the Brazilian state of the Northeast and from Ilhéus to Itacaré. Monitoring and identifying this type of comment is in line with the research of Zeng and Gerritsen (2014ZENG, B.; GERRITSEN, R. What do we know about social media in tourism? A review. Tourism Management Perspectives, [s.l.], v. 10, [s.n.], p. 27-36, Apr. 2014.) and Fotis, Buhalis, and Rossides (2012FOTIS, J.; BUHALIS, D.; ROSSIDES, N. Social media use and impact during the holiday travel planning process. In.: Fuchs, M.; Ricci, F.: Cantoni, L. Information and Communication Technologies in Tourism. Sweden: Spring, 2012. Available in: https://link.springer.com/chapter/10.1007/978-3-7091-1142-0_2. Access in: 15 mar. 2019.
https://link.springer.com/chapter/10.100...
), which reinforces the importance of social media in the search for information when making decisions. For example, comments expressing a demand for experiences in specific locations can support decisions on preparing travel packages. The analysis of 30 incidences of the word "ticket," 21 of the word "agency," and 15 of the word "ship" also pointed out some additional, although less reported, problems, such as after-sales issues, unsatisfactory service and difficulties in getting the chargeback for canceled flights. Examples are available in Table 18.
The comments containing the words "dream" and "saudade" indicate the desire of users to know the places and experiences announced by the company, as well as the feeling of longing for past experiences. These comments demonstrate successful situations in companies' content marketing efforts through posts, arousing consumers' desire for possible products and services offered by the company (FOROUZANDEH; SOLTANPANAH; SHEIKHAHMADI, 2014FOROUZANDEH, S.; SOLTANPANAH, H.; SHEIKHAHMADI, A. Content marketing through data mining on Facebook social network. Webology, [s.l.], [s.n.], n. 1, p. 1-11, 2014.). It is interesting to note that there is a possible tendency in the sentiment analysis algorithms to classify comments that express longing and desire as being negative since there is a subjectivity of the meaning of these words concerning the different contexts of a sentence.
Analyzing Table 19, we can observe that with the use of Watson's automated keyword extraction tool, it is possible to identify the same relevant words concerning the company's service aspects, identified with the automated extraction of nouns and adjectives.
4.4 Watson NLU’s Performance on Emotion Classification Test
Finally, using the IBM Watson module to analyze emotions, we present the classification performance test results for three classes of emotions (Table 20). The database contains a balanced and an unbalanced corpus. We can observe higher accuracy than the results described by Aguiar et al. (2018Aguiar, E. et al. Análise de Sentimento em Redes Sociais para a Língua Portuguesa Utilizando Algoritmos de Classificação. In.: Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, 36., 2018, Porto Alegre. Proceedings… Porto Alegre: [s.n.], 2018. p. 393-406,) using the same database, which may indicate the development of the tool's classification algorithms over the years, expanding the relevance of the method used in the present research.
5. Final considerations
Given the results presented, it is possible to state that the research achieves its proposed objectives. The analysis of the polarity of emotions identified an apparent prevalence of positive comments on the pages of the two companies, which may indicate a good level of consumer satisfaction. In addition, the results of the T-Student test indicate that there may be differences in the mean polarity of feeling when comparing the two social media while analyzing each company individually. However, the contradictory results observed while analyzing the two companies together counteract the hypothetical existence of a clear trend, specifically from the users of one of the two social networks, in the polarity of the feelings expressed.
There was also a positive correlation between the companies' online responsiveness and the polarity of feelings expressed in customer comments. However, there is a tendency for the administrators of the analyzed companies' pages to interact more with comments that express positive feelings, which may act to the detriment of a company's levels of responsiveness, reducing the positive impacts on consumer satisfaction.
The analysis of the nouns, adjectives, and other keywords extracted from the comments indicates the presence of aspects of the provided service that could be improved, particularly the ability of companies to manage unforeseen events caused by cancellations, changes in flights, and other problems of partner airlines, in order to ensure the satisfaction of its customers. In one of the companies, there is also a problem with fraud carried out by third parties using the company's name illegally, negatively influencing its operations and image.
The present work offers relevant contributions for researchers and managers, incorporating natural language processing techniques in extracting content generated by users on social media and identifying signs of problems and possible points for improvement in offered services and products. In addition, the identified trends in consumer behavior on social media can support decision-making concerning digital marketing strategies, including monitoring customer comments, while providing insights for future research on consumer behavior on social networks.
Regarding the tourism sector, we recommend that the agencies invest in better ways to manage unforeseen events caused by flight cancellations and other problems of partner airlines since this study identified great dissatisfaction about this.
Agencies can also benefit by increasing their responsiveness by instructing their social media moderators to pay greater attention to their clients' negative and neutral comments.
It is important to note that there are limitations in research inherent mainly to the technology used. First, the great difficulty in obtaining the approval of the companies that own the social media to use APIs for automated data collection made manual data collection a slow process. It limited the number of comments collected over the time available for conducting the research. Second, We can observe that there is currently much room for improvement in sentiment analysis algorithms and other natural language processing tools available for the analysis of texts in Portuguese, a fact possibly related to little effort in the creation of Portuguese text datasets labeled for the training of machine learning models and the development of updated lexical dictionaries of the language.
Given the above, we emphasize the importance of future research on the development and improvement of natural language processing tools in the Portuguese language since, as previously reported, there is an expressive adoption of social media in Brazilian daily life, which, in turn, represents a massive generation of data that can be used as research sources, increasing the positive impact not only in the productive business sectors but also in society as a whole. With concern to social media applied in the tourism sector, future research may analyze aspects such as the correlation of the polarity rates of emotions expressed in comments with the different constructs of quality and consumer satisfaction present in the literature in order to enrich the theoretical bases for the application of more studies using sentiment analysis techniques in this environment. Also, more studies comparing the two social networks when it comes to the polarity expressed in their comments may provide us with a better understanding of the average user of the two networks, as well as insights into how different characteristics in the style of posting on a specific social network may influence the polarity of the response comments, in order to assist companies’ decisions when it comes to online communication.
References
- AGNIHOTRI, R. et al. Social media: Influencing customer satisfaction in B2B sales. Industrial Marketing Management, [s.l.], v. 53, [s.n.], p. 172-180, 2016.
- Aguiar, E. et al. Análise de Sentimento em Redes Sociais para a Língua Portuguesa Utilizando Algoritmos de Classificação. In.: Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos, 36., 2018, Porto Alegre. Proceedings… Porto Alegre: [s.n.], 2018. p. 393-406,
- AMARAL, F. et al. Comentários no TripAdvisor: do que falam os turistas?. Dos Algarves: a multidisciplinary e-journal, [s.l.], v. 2, n. 26, p. 47-67, 2017.
- ARAUJO, M. et al. An evaluation of machine translation for multilingual sentence-level sentiment analysis. In.: PROCEEDINGS OF THE 31ST ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING, 31., 2016, Pisa. Proceedings… Pisa: ACM, 2016. p. 1140-1145.
- ARAÚJO, M. et al. iFeel: a system that compares and combines sentiment analysis methods. In.: THE 23RD INTERNATIONAL CONFERENCE ON WORLD WIDE WEB, 23., 2014, Seoul. Proceedings… Seoul: ACM Press, 2014. p. 75-78.
- ARAÚJO, M.; GONÇALVES, P.; BENEVENUTO, F. Measuring sentiments in online social networks. In.: PROCEEDINGS OF THE 19TH BRAZILIAN SYMPOSIUM ON MULTIMEDIA AND THE WEB, 19., 2013, Salvador. Proceedings… Salvador: ACM Press, 2013. p. 97-104.
- ARAÚJO, R. Marketing científico digital e métricas alternativas para periódicos: da visibilidade ao engajamento. Perspectivas em Ciência da Informação, Belo Horizonte, v. 20, n. 3, p. 67-84, 2015.
- BALTES, L. Content marketing-the fundamental tool of digital marketing. Bulletin of the transilvania university of brasov. Series V: economic sciences, Romania, v. 8, n. 2, p. 111-118, 2015.
- BARDIN, L. Análise de conteúdo. Lisboa: Edições, 2011. v. 70. (Edição revista e actualizada)
- BRADBURY, K. The growing role of social media in tourism marketing. In.: BRADBURY, K. Blog·bur·y. [s.l.]: [s.n.], 2011. Avaliable in: https://kelseybradbury.weebly.com/uploads/1/0/9/2/10927387/tourismsocialmedia-comm427.pdf Access in: 07 jun. 2019.
» https://kelseybradbury.weebly.com/uploads/1/0/9/2/10927387/tourismsocialmedia-comm427.pdf - BRANCO, L.; CAVALCANTI, G.; DOCA, G. Avianca Brasil pede recuperação judicial por risco de paralisar suas operações. O Globo, Rio de Janeiro, [s.n.], [s.n.], 17 abr. 2019. Economia Available in: https://oglobo.globo.com/economia/avianca-brasil-pede-recuperacao-judicial-por-risco-de-paralisar-suas-operacoes-23297762 Access in: 07 jun. 2019.
» https://oglobo.globo.com/economia/avianca-brasil-pede-recuperacao-judicial-por-risco-de-paralisar-suas-operacoes-23297762 - CHOI, J.; TETREAULT, J.; STENT, A. It depends: dependency parser comparison using a web-based evaluation tool. In.: PROCEEDINGS OF THE 53RD ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, 58.; THE 7TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING, 7., 2015, Beijing. Proceedings… Beijing: ACL Anthology, 2015. p. 387-396. (Volume 1: Long Papers).
- COELHO, M. F.; GOSLING, M. S. Comentar bem ou mal na internet? O engajamento de viajantes em reviews de hotéis. Revista: Turydes Revista Turismo y Desarrollo2015.
- DWIVEDI, M.; SHIBU, T.; VENKATESH, U. Social software practices on the Internet: Implications for the hotel industry. International journal of contemporary hospitality management, [s.l.], v. 19, n. 5, p. 415-426, 2007.
- EVANGELISTA, T.; PADILHA, T. Monitoramento de postagens sobre empresas de e-commerce em redes sociais utilizando análise de sentimentos. In.: BRAZILIAN WORKSHOP ON SOCIAL NETWORK ANALYSIS AND MINING (BraSNAM), 2., 2014, Porto Alegre. Proceedings… Porto Alegre: SBCOPENLIB, 2014.
- FOROUZANDEH, S.; SOLTANPANAH, H.; SHEIKHAHMADI, A. Content marketing through data mining on Facebook social network. Webology, [s.l.], [s.n.], n. 1, p. 1-11, 2014.
- FOTIS, J.; BUHALIS, D.; ROSSIDES, N. Social media use and impact during the holiday travel planning process. In.: Fuchs, M.; Ricci, F.: Cantoni, L. Information and Communication Technologies in Tourism. Sweden: Spring, 2012. Available in: https://link.springer.com/chapter/10.1007/978-3-7091-1142-0_2 Access in: 15 mar. 2019.
» https://link.springer.com/chapter/10.1007/978-3-7091-1142-0_2 - GALLAUGHER, J.; RANSBOTHAM, S. Social media and customer dialog management at starbucks. MIS Quarterly Executive, [s.l.], v. 9, n. 4, 2010.
- GONÇALVES, P. et al. Comparing and combining sentiment analysis methods. In.: PROCEEDINGS OF THE FIRST ACM CONFERENCE ON ONLINE SOCIAL NETWORKS, 1., 2013, [s.l.]. Proceedings… [s.l.]: ACM Press, 2013.
- GONZALES, M.; LIMA, V. Recuperação de informação e processamento da linguagem natural. In.: Congresso da sociedade Brasileira de computação, 23., 2003, [s.l.]. Proceedings… [s.l.]: [s.n.], 2003. p. 347-395
- HEARST, M. Untangling Text Data Mining. In.: PROCEEDINGS OF THE 37TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS ON COMPUTATIONAL LINGUISTICS, 37., 1999, College Park. Proceedings… College Park: University of Maryland, 1999. p. 3-10.
- LEUNG, D. et al. Social media in tourism and hospitality: a literature review. Journal of Travel & Tourism Marketing, [s.l.], v. 30, n. 1-2, p. 3-22, jan. 2013.
- LIMA, V. M. Engajamento do consumidor em uma comunidade virtual de marca. 2014. 103 f. Dissertação (Mestrado em Gestão Empresarial) - Escola Brasileira de Administração Pública e de Empresas, Fundação Getulio Vargas, Rio de Janeiro, 2014.
- LUCAS, A. S. Top 10 maiores agencias de viagens do Brasil. [s.l.]: Top10+, c2013. (Luxo). Available in: https://top10mais.org/top-10-maiores-agencias-de-viagens-brasil/. Access in: 15 mar. 2019.
» https://top10mais.org/top-10-maiores-agencias-de-viagens-brasil - MALHOTRA, N. Pesquisa de marketing: uma orientação aplicada. 3. ed. Porto Alegre: Bookman, 2001.
- MIRANDA, M. D.; SASSI, R. J. Using sentiment analysis to assess customer satisfaction in an online job search company. Business Information Systems Workshops. [s.l.], v. 183, [s.n.], p. 17-27, 2014.
- MOGHADDAM, S.; ESTER, M. Opinion digger: an unsupervised opinion miner from unstructured product reviews. In.: PROCEEDINGS OF THE 19TH ACM INTERNATIONAL CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, 19., 2010, Toronto. Proceedings… Toronto: ACM Digital Library, 2010, p. 1825-1828.
- MORENO, A. C. Análise de sentimentos na classificação de comentários online aplicando técnicas de text mining. 2015, 72 f. Dissertação (Mestrado em Sistemas Integrados de Apoio à Decisão) - Departamento de Ciências e Tecnologias de Informação, Instituto Universitário de Lisboa, Lisboa, 2015.
- PANG, B.; LEE, L. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, [s.l.], v. 2, n. 1-2, p. 1-135, 2008.
- PANTELIDIS, I. Electronic meal experience: a content analysis of online restaurant comments. Cornell Hospitality Quarterly, [s.l.], v. 51, n. 4, p. 483-491, ago. 2010.
- REIS, J. et al. Uma abordagem multilıngue para análise de sentimentos. In.: BRAZILIAN WORKSHOP ON SOCIAL NETWORK ANALYSIS AND MINING (BraSNAM), 4., 2015, Porto Alegre. Proceedings… Porto Alegre: SBCOPENLIB, 2015.
- RIBEIRO, F. N. et al. Sentibench - a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Science, [s.l.], v. 5, n. 23, 2016.
- SANTOS, G. C. O. et al. As redes sociais e o turismo: uma análise do compartilhamento no Instagram do Festival Cultura e Gastronomia de Tiradentes. Revista Iberoamericana de Turismo (RITUR), Penedo, v. 7, n. 2, p. 60-85, 2017
- SCHULTZ, B. B. Levene's test for relative variation. Systematic Zoology, [s.l.], v. 34, n. 4, p. 449-456, dec. 1985.
- SHARDA, R.; DELEN, D.; TURBAN, E. Business intelligence: a managerial perspective on analytics. Hoboken: Prentice Hall Press, 2013.
- SILVA, C.; TESSAROLO, F. Influenciadores digitais e as redes sociais enquanto plataformas de mídia. In.: CONGRESSO BRASILEIRO DE CIÊNCIAS DA COMUNICAÇÃO, 39., 2016, São Paulo. Proceedings… São Paulo: INTERCOM, 2016.
- SOUSA, A. R. Processamento automático de línguas naturais: Um estudo sobre a localização do IBM Watson™ para o português do Brasil. 2017. 76 f. Monografia (Bacharel em Línguas Estrangeiras Aplicadas ao Multilinguismo e à Sociedade da Informação) - Instituto de Letras, Universidade de Brasília, Brasília (DF), 2017.
- STICH, V.; EMONTS-HOLLEY, R.; SENDEREK, R. Social media analytics in customer service: a literature overview - an overview of literature and metrics regarding social media analysis in customer service. In.: 11TH INTERNATIONAL CONFERENCE ON WEB INFORMATION SYSTEMS AND TECHNOLOGIES, 11., 2015, [s.l.], Proceedings… [s.l.]: SCITEPRESS - Science and Technology Publications, 2015.
- THEVENOT, G. Blogging as a social media. Tourism and hospitality research, [s.l.], v. 7, n. 3-4, p. 287-289, 2007.
- THINK with Google. The 2012 Traveler. USA: Google: Ipsos MediaCT, 2012. Available in: https://www.thinkwithgoogle.com/_qs/documents/682/the-2012-traveler_research-studies.pdf Access in: 11 ago. 2019.
» https://www.thinkwithgoogle.com/_qs/documents/682/the-2012-traveler_research-studies.pdf - VICENTE, M. Uma imagem vale mais que mil palavras? Humans Of New York e a febre de páginas que contam histórias de anônimos através de fotografias no Facebook. Revista Mangaio Acadêmico, [s.l.], v. 1, n. 1, p. 01-11, 2016.
- YE, Q.; LAW, R.; GU, B. The Impact of Online User Reviews on Hotel Room Sales. International Journal of Hospitality Management, [s.l.], v. 28, n. 1, p. 180-182, mar. 2009.
- ZENG, B.; GERRITSEN, R. What do we know about social media in tourism? A review. Tourism Management Perspectives, [s.l.], v. 10, [s.n.], p. 27-36, Apr. 2014.
-
1
User name censored for privacy.
-
2
User name censored for privacy.
-
3
User name is censored for privacy.
-
4
There is no corresponding English word for the portuguese word “saudades", which means the longing for something that is in the past.
-
5
User name censored for privacy.
-
6
User name censored for privacy.
-
7
User name censored for privacy.
-
8
The portuguese word “gente” could be translated to English both as "people" or as "us," depending on the context.
APPENDIX
Script operation in Facebook spreadsheets: (1) The rows whose following lines contain values in the "answer" column are searched. (2) Once the script finds a line, the number of answers in the comment is counted until it reaches the following line that does not contain the cited column. (3) The search for comments from the company's official user is carried out on these lines. If it finds multiple recurrences of company comments, it extracts from each company comment the list of entities in the text. (4) Each entity in this list is searched in the answers users column to search for the answered user. (5) These data are saved in a row of another spreadsheet containing: the comment, the polarity of emotions (according to the Sentistrength and Umigon methods), and the company's response. It repeats the procedure on the following line, non-classified as an answer.
Script operation in Instagram spreadsheets: (1) Comments in a post are counted. (2) A search is performed for each line in company comments on a post. (3) Once it identifies the company's comment, the words that start with the character '@' 'are searched in the text since it contains the answered user's name. (4) Each word found is searched in the post's user list. (5) If it finds a user, these data are saved in a row of another spreadsheet containing: the comment, the polarity of emotions (according to the Sentistrength and Umigon methods), and the company's response. It repeats the procedure on the following line, not classified as an answer.
Publication Dates
-
Publication in this collection
24 July 2023 -
Date of issue
2023
History
-
Received
03 Apr 2023 -
Accepted
11 May 2023