ACTIVITY OF BRAZILIAN TOURISM AGENCIES IN SOCIAL MEDIA: AN ANALYSIS USING NATURAL LANGUAGE PROCESSING

Guedes, Danillo Magno Duarte; Gosling, Marlusa de Sevilha

doi:10.1590/1981-5344/25280

Sumário Anterior Atual Seguinte

Sumário

Artigos • Perspect. ciênc. inf. 28 • 2023 • https://doi.org/10.1590/1981-5344/25280 linkcopy

ACTIVITY OF BRAZILIAN TOURISM AGENCIES IN SOCIAL MEDIA: AN ANALYSIS USING NATURAL LANGUAGE PROCESSING

ATIVIDADE DE AGÊNCIAS BRASILEIRAS DE TURISMO NAS REDES SOCIAIS: UMA ANÁLISE UTILIZANDO PROCESSAMENTO DE LINGUAGEM NATURAL

Authorship SCIMAGO INSTITUTIONS RANKINGS

This paper aims to analyze the activity of two Brazilian tourism agencies in social media and the online behavior of their consumers. The research used Natural language processing resources supported by sentiment and content analysis techniques. The main results show a prevalence of positive comments on the companies' pages, and companies are more responsive to users on pages with a higher number of positive comments. There is also a tendency towards more significant company interaction with user comments that express positive emotions.

Keywords:
Content marketing; Social media; User-generated content; Natural language processing; Sentiment analysis

Authors	Results
Dwivedi et al. (2007)	The Authors indicate an imminent loss of control over the network's information regarding companies and industries.
Thevenot (2007)	The study concluded that if companies would not properly monitor online comments, the industry would bear the consequences, as blogs generate positive and negative impacts.
Blair-Goldensohn et. al. (2008)	The authors utilized Google Maps data to construct the sentiment analysis tool to analyze consumers' emotions regarding hotels, stores, and restaurants, determining the value of the polarities (positive/negative). In addition, the system summarized emotions concerning different aspects of the services provided, such as price and environment.
Pekar and Ou (2008)	The authors use Sentiment analysis to evaluate 268 customer reviews of large hotels published on the 22epinions.com website. The authors use resources such as food, room service, facilities, and price to analyze automatically the emotions expressed regarding each hotel.
Barcelos et al. (2015)	The results demonstrate how the categorization by themes and data visualization in graphs can condense the information available on a social media platform specialized in tourism (TripAdvisor).
Bakhshi et al. (2015)	The study identifies that the most active and regular members of the community are the ones that most contribute to the good quality of the comments and that the most extended comments are more likely to be popular, receiving stars and votes. Also, consumers tend to see useful comments first.
Geetha and Sinha (2017)	The authors identify consistency between the customer ratings and their real emotions regarding hotels of both categories of, premium and budget. The polarity of customer emotions explains the significant variation in customer ratings in both hotel categories.

Specific objective	Analyzed Units	Data collection	Data Analysis
Analyze the emotions expressed in the comments on the posts	Company A (Facebook: 7536 comments on 221 posts from 04/2018 to 04/2019. Instagram: 7907 comments on 98 posts from 07/2018 to 04/2019). Company B (Facebook: 8344 comments on 46 posts from 11/2018 to 04/2019. Instagram: 7917 comments on 76 posts from 01/2019 to 04/2019).	Manual Collection. Automated data structuring using the Python language.	Tools: Sentiment Analysis - iFeel and IBM Watson NLU Metrics: P/NG, P/Total, NG/Total, P/(P + N), NG/(P+N)
Investigate whether there is a difference in the emotion expressed in posts from different social media.			Tools: T-Student Test Metrics: average of sentiment polarity
Investigate the degree of companies' responsiveness regarding complaints, suggestions, and other manifestations of their users.			Tools: Response Index Metrics: Responses to negative comments/total responses
Look for signs of service aspects with potential for improvement.			Content analysis, Natural Language Processing (spaCY Library and IBM Watson NLU)

	Company A		Company B
	Instagram	Facebook	Instagram	Facebook
Total posts	90	216	76	44
Total followers comments	6677	5631	6498	5365
The average number of comments per post	74,19	26,06944	85,5	121,19
Percentage of company comments	15,55%	25,28%	17,92%	35,7%

	Method	Positive	Negative	Neutral	P/NG	P/Total	NG/Total	P/(P+NG)	NG/(P+NG)
	SentiSTR	2142	613	3922	3,494	0,320	0,0918	0,7774	0,2225
Total	Watson	3779	1340	1741	2,820	0,5508	0,1953	0,73823	0,26177
Total	Umigon	1525	843	4309	1,809	0,2283	0,1262	0,644	0,3559
Average per post	SentiSTR	23,8	6,81111	43,57	5,430	0,299	0,1311	0,7222	0,2777
Average per post	Watson	41,988	14,8888	19,34	4,624	0,530	0,2435	0,70105	0,2989
	Umigon	16,944	9,36666	47,87	2,694	0,216	0,1763	0,5814	0,4074

	Method (+NLU)	Positive	Negative	Neutral	P/NG	P/Total	NG/Total	P/(P+NG)	NG/(P+N)
Total	SentiSTR	1384	418	3829	3,311	0,24578	0,074232	0,768	0,2319
	Watson	2133	1000	2379	2,133	0,386974	0,181422	0,6808	0,3191
	Umigon	884	528	4219	1,6742	0,15698	0,093767	0,626	0,3739
Average per post	SentiSTR	6,4074	1,9351	17,72	3,7385	0,25559	0,069246	0,7064	0,1916
	Watson	9,7397	4,5662	10,86	3,43348	0,373987	0,162645	0,6468	0,27552
	Umigon	4,0925	2,4444	19,53	2,0816	0,16741	0,085797	0,5678	0,2979

	Sentistrengh Method		Umigon Method
	Facebook	Instagram	Facebook	Instagram
Mean	0,086333	0,210058	-0,01281	0,08747
Variance	0,340118	0,357453	0,292697	0,335977
Observations	3278	6184	3279	6185
P-value:	0,0000		0,0000
Df	6823		7090
t Stat	-9,73396		-8,36859
P(T<=t) one-tail	1,51E-22		3,48E-17
t Critical one-tail	1,645077		1,645069
P(T<=t) two-tail	3,01E-22		6,96E-17
t Critical two-tail	1,960312		1,960299

	Method (+NLU)	Positive	Negative	Neutral	P/NG	P/Total	NG/Total	P/(P+NG)	NG/(P+N)
Total	SentiSTR	2033	283	4182	7,183	0,312	0,0435	0,8778	0,122193
	Watson	1651	437	1563	3,7780	0,452	0,1196	0,79070	0,209291
	Umigon	1168	397	4933	2,942	0,179	0,061	0,74632	0,253674
Average per post	SentiSTR	26,75	3,72368	55,02	10,47	0,314	0,0539	0,85873	0,128108
	Watson	9,7397	4,56621	10,86	3,433	0,373	0,16264	0,64684	0,275526
	Umigon	15,368	5,22368	64,90	3,718	0,196	0,0772	0,7153	0,271539

	Method (+NLU)	Positive	Negative	Neutral	P/NG	P/Total	NG/ Total	P/(P+NG)	NG/(P+N)
Total	SentiSTR	2038	156	3171	13,06	0,379	0,029	0,9288	0,0711
	Watson	3178	362	1727	8,779	0,603	0,0687	0,89774	0,10226
	Umigon	1580	299	3486	5,284	0,294	0,0557	0,8408	0,15912
Average per post	SentiSTR	46,318	3,5454	72,06	12,16	0,362	0,0303	0,9199	0,08009
	Watson	72,227	8,22727	39,25	11,70076	0,580	0,0737	0,88588	0,114111
	Umigon	35,909	6,7954	79,22	6,115	0,259	0,0456	0,8466	0,1533

	Method Sentistrengh		Method Umigon
	Facebook	Instagram	Facebook	Instagram
Mean	0,350726	0,258425	0,238471	0,110473
Variance	0,290236	0,300498	0,29777	0,241573
Observations	5443	6617	5443	6617
P-value:	0,0000		0,0000
Df	0		0
t Stat	11686		11073
P(T<=t) one-tail	9,288946		13,40207
t Critical one-tail	9,16E-21		6,09E-41
P(T<=t) two-tail	1,644984		1,644991
t Critical two-tail	1,83E-20		1,22E-40

	Company A		Company B
	Instagram	Facebook	Instagram	Facebook
Total answered comments	1035	769	1262	1403
Total answered people	981	631	1249	1136
Percentage of answered comments	15,50%	13,65%	19,42%	26,15%

	Method	Positive	Negative	Neutral	P/N	P/Total	NG/Total	P/(P+NG)	NG/(P+NG)
Comments	SentiSTR	571	59	405	9,6779	0,55169	0,057	0,9063	0,09365
Comments	Umigon	460	103	472	4,4660	0,44444	0,099	0,8170	0,18294
People	SentiSTR	558	58	365	9,6206	0,56881	0,059	0,9058	0,09415
People	Umigon	451	101	429	4,4653	0,45974	0,103	0,8170	0,18297

	Method	Positive	Negative	Neutral	P/N	P /total	NG /total	P/(P+NG)	NG/(P+NG)
Comments	SentiSTR	387	31	351	12,483	0,50325	0,0403	0,9258	0,07416
Comments	Umigon	332	80	357	4,15	0,43173	0,1040	0,8058	0,19417
People	SentiSTR	354	20	257	17,7	0,56101	0,0316	0,9465	0,05347
People	Umigon	309	58	264	5,3275	0,48969	0,0919	0,8419	0,15803

	Method	Positive	Negative	Neutral	P/N	P /total	NG /total	P/(P+NG)	NG/(P+NG)
Comments	SentiSTR	519	46	697	11,282	0,41125	0,0364	0,9185	0,08141
Comments	Umigon	313	84	865	3,7261	0,24801	0,0665	0,7884	0,21158
People	SentiSTR	354	20	257	17,7	0,56101	0,0316	0,9465	0,05347
People	Umigon	309	58	264	5,3275	0,48969	0,0919	0,8419	0,15803

	Method	Positive	Negative	Neutral	P/N	P /total	NG /total	P/(P+NG)	NG/(P+NG)
Comments	SentiSTR	754	29	620	26	0,53742	0,0206	0,9629	0,03703
Comments	Umigon	720	87	596	8,2758	0,51318	0,0620	0,8921	0,10780
People	SentiSTR	354	20	257	17,7	0,56101	0,0316	0,9465	0,05347
People	Umigon	309	58	264	5,3275	0,48969	0,0919	0,8419	0,15803

Word	Incidence	Associated Adjectives
company	195	terrible, serious, worst, horrible, crap, garbage, biggest, reliable, crook, aerial, successful, corrupt, cancel, irresponsible, dignified, honest
ticket	162	expensive, cheap, problematic, integral, failed, area, super, aerial
problem	134	@user¹, very serious, serious, impossible, unique, boring, ridiculous
Site	104	parallel, difficult, loud, pitiful
trip	104	next, aerial, whole, good, direct, first
tickets	95	aerial, guys, good
money	87	@user²
purchase	82	Final, urgent
service	81	terrible, sale, terrible, horrible, decent, easy, bad, different, worst
package	80	cheap, sinister
contact	73	with me, difficult, urgent
customer	48	neglect (with), direct, available
reservation	46	urgent
telephone	42	App
flight	42	Canceled

Problem	Example of comment extracted from the page
ticket not sent	please I bought 3 tickets to João Pessoa by bank slip, paid on 12/03 and I still have had no answer so I'm desperate help me please, I can't buy another ticket ??
	I was blocked on Messenger where I handled tickets, for questioning other fees they said they forgot to launch.
	(Company reply) User³, we do not make sales through Facebook Messenger or any other social media. Only through the website, app or telesales at the number available on our website.
difficulty of canceling	I had a death in the family and you didn’t cancel the ticket
cancelled ticket	Dear company A, I came to inform you that I have MY WEDDING in the city of Fortaleza on April 28th, and many of my guests bought tickets from your company on the GRU - FOR route to attend the event. INCLUDING MY PARENTS !! The flights were canceled and you did not guarantee relocation. (...)
lack of refund	Too bad I made a purchase, canceled within 24 hours and so far I have not received a refund! Shame on you @company A owing me a refund of R$ 6 thousand over 1 month !!!!!
expensive ticket	I bought a ticket from Acre to Santa Catarina, I paid R $ 851.03 I think the price is very high, I have friends who bought from other companies much more cheaply, next time I will do more research to avoid falling into this "trap".
difficulty on buying tickets	I can’t buy tickets on the website.
difficulty on buying tickets	Company A's website has been redone to confuse the customer. Simply horrible!!!!!
rescheduling fee	Watch out people! Company with terrible service We changed the trip's date, it was imposed a charge of almost R $ 500.00 of fines and fees and when the e-mail with the exchange confirmation came, the date was wrong We opened a complaint at the company and they said that the conversation indicated that the mistake was ours (...)

Word	Incidence	Associated adjectives
trip	76	first, next, good, aerial, great, unforgettable, special
package	53	expensive, complete, neglect
dream	44	old, cool, silly, big
company	41	aerial, Avianca, garbage, best, worst, single, partner, show, dissatisfied
problem	37	urgent
passage	30	aerial, expensive, complete
place	24	full, good, expensive, spectacular
service	23	best, terrible, bad
saudades⁴	23	guide, girls, unforgettable
agency	21	worst terrible, multi-brands, place, best
fear	20	now
ship	15	sovereign, good, old
packages	14	new
daughter	14
cruise	14	cheap, awesome
money	14	lack, lost
company	13	lack
city	12	small, worst, big
hotel	12	luxurious
valor	11	final, high

Problem	Example of comment extracted from the page
canceled or rescheduled flights	@user⁵ My trip was canceled the day before departure 04/27. I was supposed to embark today 04/28. I don't want to discourage you, but if they are negligent with you like they were with me, your trip will be canceled.
	I’ve been scheduling the vacation trip for 3 months, the day before departure you cancel my trip and just ask to go to the agency for a refund. It is a total disregard for the client who trusted you. Now all there is to do is unpacking. sitting and crying, because my dream of visiting another place has been canceled. Thank you for making my long-awaited vacation the worst of all.
	Company B my flight was canceled, my trip is 05/05, what do I do?
desire to travel	A dream, an unforgettable trip, only if sponsored by company B, because I have no conditions 😢
fear for negative comments	Mercy, I read so many negative comments that I will no longer be choosing company B to provide my trip.
problems with Avianca	@user⁶ did you get on board? I bought 4 packages with company B for June 30th and the flights are from Avianca. I have already contacted them, as I don't want to arrive at the airport and just go back, specially with 2 small children
problems on after sales service	Bad after sales service ... NEVER AGAIN.
No chargeback	Solve my problem and refund my money

Company A	Incidence	Company B	Incidence
Money	200	company B	59
User⁷	180	Company B(id)	28
Ticket	118	package	26
Company	106	dream	23
Problem	78	to	15
Contact	77	gente⁸	15
You	76	problem	15
Tickets	76	ticket	13
Purchase	65	company	13
Package	62	problems	13
Site	60	desire	12
Client	47	money	11
Telefone	44	people	11
No	44	promotion	10

Corpus	Total Successes	Errors	Accuracy	Precision	Recall	F1-Measure
unbalanced	1257	762	0.622585438336	0,655864	0,776	0,70693
balanced	317	181	0.636546184739	0,545664	0,702992	0,565697

location_on

Escola de Ciência da Informação da UFMG Antonio Carlos, 6627 - Pampulha, 31270- 901 - Belo Horizonte -MG, Brasil, Tel: 031) 3499-5227 , Fax: (031) 3499-5200 - Belo Horizonte - MG - Brazil
E-mail: pci@eci.ufmg.br

rss_feed Acompanhe os números deste periódico no seu leitor de RSS

[TFN1] Source: prepared by the authors (2019) based on Moreno (2015).