Acessibilidade / Reportar erro

Paper-based thesis and dissertations: analysis of fundamental characteristics for achieving a robust structure

Abstract

Paper aims

This study identifies fundamental characteristics for developing a paper-based thesis (PBT), providing some recommendations to researchers who decide for this academic research report.

Originality

This paper contributes to the literature by providing guidance to develop PBTs, which has been attracting researchers' interest because it demonstrates publication capacity and potential as a researcher. Additionally, it is still not entirely clear which issues and countermeasures researchers should address when choosing the paper-based structure.

Research method

The authors searched for articles of PBTs already published, institutional documents (e.g., regulations for conducting PBTs in Brazil and abroad) and journal papers concerning this subject. A content and document analysis were conducted in the papers and PBT internal regulations, respectively.

Main findings

Results indicate five main factors that impact the success of a PBT approach: (i) the initial planning for this academic model; (ii) alignment between the articles and the thesis/dissertation; (iii) research design construction; (iv) copyrights requirements; and (v) issues regarding co-authorship. Based on that, the study also builds some guidelines to structure a robust PBT.

Implications for theory and practice

Our recommendations might be meaningful to scholars and researchers bring more theoretical, empirical, and structural robustness when developing a PBT final document.

Keywords
Article-based thesis; Publication-based thesis; Thesis by publications; Thesis as a series of papers; Hybrid thesis

1. Introduction

Pressure for publications has been growing in the academic environment. Consequently, there is also an increasing interest in the development of the paper-based thesis (PBT); this interest is growing because of the competition in the academic research sector, which is continuously rising, with more candidates disputing for decreasing funding resources (Rowland, 2017Rowland, J. (2017). Practical academic: delivering a thesis by publication. United States: Practical Academic.; Jowsey et al., 2019Jowsey, T., Corter, A., & Thompson, A. (2019). Are doctoral theses with articles more popular than monographs? Supervisors and students in biological and health sciences weigh up risks and benefits. Higher Education Research & Development, 39(4), 719-732. http://dx.doi.org/10.1080/07294360.2019.1693517.
http://dx.doi.org/10.1080/07294360.2019....
). Australia, South Africa, and some European countries (e.g., England and Ireland) have led this process, while this is an uncommon model in the United States of America and Canada (Robins & Kanowski, 2008Robins, L., & Kanowski, P. (2008). PhD by publication: a student’s perspective. Journal of Research Practice, 4(2), M3. Retrieved in 2020, September 17, from http://jrp.icaap.org/index.php/jrp/article/view/136/154
http://jrp.icaap.org/index.php/jrp/artic...
; Haq et al., 2016Haq, K., Azariadis, M., & Edmondston, J. (2016). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/__data/assets/pdf_file/0006/2930217/ThesisByPapers_160916.pdf
http://www.postgraduate.uwa.edu.au/__dat...
; Mason, 2018Mason, S. (2018). Publications in the doctoral thesis: challenges for doctoral candidates, supervisors, examiners and administrators. Higher Education Research & Development, 37(6), 1231-1244. http://dx.doi.org/10.1080/07294360.2018.1462307.
http://dx.doi.org/10.1080/07294360.2018....
; Frick, 2019Frick, L. (2019). PhD by publication: panacea or paralysis? Africa Education Review, 16(5), 47-59. http://dx.doi.org/10.1080/18146627.2017.1340802.
http://dx.doi.org/10.1080/18146627.2017....
). There is an increasing impetus to publish during doctoral and master’s candidature, and research performance commonly determines university funding and doctorates with publishing experience become better positioned (Jackson, 2013Jackson, D. (2013). Completing a PhD by publication: a review of Australian policy and implications for practice. Higher Education Research & Development, 32(3), 355-368. http://dx.doi.org/10.1080/07294360.2012.692666.
http://dx.doi.org/10.1080/07294360.2012....
); i.e., publication output considerably affects academic selection and promotion, access to further research funding and professional opportunities (Kamler, 2008Kamler, B. (2008). Rethinking doctoral publication practices: writing from and beyond the thesis. Studies in Higher Education, 33(3), 283-294. http://dx.doi.org/10.1080/03075070802049236.
http://dx.doi.org/10.1080/03075070802049...
). Accordingly, the PBT has been attracting interest and gaining importance for contributing to the competitiveness and recognition by the researchers along the time (Taylor, 2001Taylor, J. (2001). The impact of performance indicators on the work of university academics: evidence from Australian universities. Higher Education Quarterly, 55(1), 42-61. http://dx.doi.org/10.1111/1468-2273.00173.
http://dx.doi.org/10.1111/1468-2273.0017...
; Robins & Kanowski, 2008Robins, L., & Kanowski, P. (2008). PhD by publication: a student’s perspective. Journal of Research Practice, 4(2), M3. Retrieved in 2020, September 17, from http://jrp.icaap.org/index.php/jrp/article/view/136/154
http://jrp.icaap.org/index.php/jrp/artic...
; Pretorius, 2017Pretorius, M. (2017). Paper-based theses as the silver bullet for increased research outputs: first hear my story as a supervisor. Higher Education Research & Development, 36(4), 823-837. http://dx.doi.org/10.1080/07294360.2016.1208639.
http://dx.doi.org/10.1080/07294360.2016....
).

A paper-based thesis consists of a research work at the master's or doctoral level, presented through a set of academic articles (submitted, accepted for publication, or published) developed by the candidate. This document embeds a comprehensive text about a research as a whole; nevertheless, several models are being adopted, according to the criteria established in each institution (Peacock, 2017Peacock, S. (2017). The PhD by publication. International Journal of Doctoral Studies, 12, 123-135. http://dx.doi.org/10.28945/3781.
http://dx.doi.org/10.28945/3781...
). There are different taxonomies utilized for PBT (Haq et al., 2016Haq, K., Azariadis, M., & Edmondston, J. (2016). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/__data/assets/pdf_file/0006/2930217/ThesisByPapers_160916.pdf
http://www.postgraduate.uwa.edu.au/__dat...
): (i) thesis as a series of papers/articles; (ii) thesis as a collection of papers/articles; (iii) thesis by papers/articles; (iv) thesis by publications; (v) integrated format; (vi) hybrid thesis; and (vii) continental model.

However, it is not yet entirely clear which issues and countermeasures academics should address when opting for preparing a PBT. The lack of studies pointing out recommendations for PBT may imply in scientifically limited academic works when they are developed in this structure (Peacock, 2017Peacock, S. (2017). The PhD by publication. International Journal of Doctoral Studies, 12, 123-135. http://dx.doi.org/10.28945/3781.
http://dx.doi.org/10.28945/3781...
). Consequently, there is still an underperformance in terms of publication output as well as in the developed paper-based thesis (Jackson, 2013Jackson, D. (2013). Completing a PhD by publication: a review of Australian policy and implications for practice. Higher Education Research & Development, 32(3), 355-368. http://dx.doi.org/10.1080/07294360.2012.692666.
http://dx.doi.org/10.1080/07294360.2012....
). The previous author underscores that workload pressures, lack of confidence (by candidates), poor infrastructure, and lack of motivation are some of the reasons for Australian academics lacking robust results in publishing. Such motifs are similar to what occurs in other contexts, such as in emerging economies like Brazil.

As argued by Haq et al. (2016)Haq, K., Azariadis, M., & Edmondston, J. (2016). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/__data/assets/pdf_file/0006/2930217/ThesisByPapers_160916.pdf
http://www.postgraduate.uwa.edu.au/__dat...
, there is no global consensus on this model of thesis or dissertation and considering only the Australian scenario – where they conducted their research – there is a wide variety of approaches in universities regarding academic research in the paper-based model. For instance, in Brazil, paper-based thesis still raises questions regarding how to carry it out, as well as concerns about the structure of the final document in terms of consistency, co-authorship (with other candidates and the supervisor), and copyright permissions. Based on those arguments, this study formulated the following research question:

  • How to build a more cohesive, coherent, and robust paper-based thesis (PBT) in terms of structure, consistency, and quality?

This study then examines graduate programs and universities regulations for PBT. It also points out the main characteristics and requirements for a robust and well-structured PBT by offering some recommendations to researchers who will choose this model to build their research final documents, and then contributing to filling that gap. Moreover, this work identifies possible benefits and challenges in conducting PBT. Next, section 2 describes the research methods and procedures adopted to carry out the study. Then, section 3 presents the results of this work, followed by a discussion of the investigated contents. Finally, section 4 draws the main concluding remarks, limitations, and future research possibilities.

2. Research methods

This section divides the methodological procedures to develop this work into methods for data collection and data analysis sub-sections.

2.1. Procedures for data collection

To conduct this study, the authors focused mainly on secondary data, initially by searching for national and international institutions, as well as graduate programs, which have specific standards and internal documents concerning dissertation and thesis writing. Scopus database was then searched by using strings related to Haq et al. (2016)Haq, K., Azariadis, M., & Edmondston, J. (2016). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/__data/assets/pdf_file/0006/2930217/ThesisByPapers_160916.pdf
http://www.postgraduate.uwa.edu.au/__dat...
's taxonomies (e.g., ‘paper-based thesis’; ‘article-based thesis’; ‘publication-based thesis’; ‘thesis by publications’; ‘thesis as a series of papers’; ‘thesis by publication’; ‘hybrid thesis’; and others).

All keywords abovementioned were combined with ‘guidelines’ (using AND as logical operator), as follows: “guidelines” AND (“paper-based thesis” OR “article-based thesis” OR “thesis by publication” OR “thesis as a series of papers” OR “thesis as a collection of papers” OR “thesis by papers” OR “hybrid thesis”). When using this exact search string, one of the authors found out only four documents (three articles and one book, with no access of its content). Then, when removing the “guidelines” keyword, search resulted in 25 documents.

The authors used the same strings to search for documents in Google in order to find standards and regulations that offer guidelines for paper-based thesis in Brazilian and overseas universities, to examine what are the most fundamental requisites when publishing a PBT. After filtering the search results, for PBT regulations, the authors retrieved 30 different guidelines from many Brazilian and overseas universities, graduate, and graduate schools. The authors examined those regulations exhaustively until no more novel evidence could be found both in guidelines and papers. Table 1 summarizes the results of each document search in Scopus.

Table 1
Search results in the Scopus database (conducted in April 2020).

In the case of national graduate programs, the authors firstly sought regulations that provided guidelines for paper-based thesis, focusing on production engineering courses and, subsequently, engineering courses in general. Secondly, the researchers expanded the search prospecting for academic documents, guidelines, and standards from other areas. This search broadening was carried out because not all graduate or graduate engineering programs have specific documentation for PBT. These programs usually follow the general normative guidelines of their institutions. Moreover, to expand the analysis, the authors recovered 30 internal regulations of graduate programs overseas and in Brazil, aiming at comparing the Brazilian context with what the international institutions establish, towards a more consistent body of knowledge to outline relevant recommendations.

2.2. Data analysis

After data collection, a content analysis of the documents was performed, particularly in the articles retrieved on the subject (PBT), and a document analysis in the internal regulations of the institutions with undergraduate and graduate programs. The study carried out the content analysis following combined guidelines by Miles & Huberman (2014)Miles, M. B., & Huberman, A. M. (2014). Qualitative data analysis. Los Angeles: SAGE. and Bardin (1977)Bardin, L. (1977). Content analysis. Lisboa: Edições 70., namely:

  • Organization of the recovered documents: initially, researchers separated data into scientific articles and resolutions/guidelines from graduate programs that offer the option of paper-based thesis. Then, it was possible to conduct a more robust analysis to separately identify what each type of document presented as a basis for PBT;

  • Data analysis and categorization: this step separated the results and considerations/recommendations found out in articles and resolutions in Brazil and abroad;

  • Inference: based on the evidence found out in literature and institutions' regulations, the authors inferred the fundamental aspects of this structural model, proposing recommendations when developing PBT. Those guidelines assess contents (e.g., features related to the introduction, methods, and conclusions) and structure issues, which refer to concatenating the parts of the final thesis document.

Having described the research methods, attention is turned to the findings presented next.

3. Results and discussion

From the analysis of the internal regulations of graduate programs in production engineering and other engineering courses, this work notes implications on the decision to conduct a PBT. The next sections present these implications and recommendations.

3.1. Thesis by publication: planning and structuring

The first relevant aspect to consider is the initial decision regarding the thesis model. When planning a research, whether at the master's or doctoral level, it is essential to define as early as possible (preferably, in the first months of research) if the candidate will conduct the PBT structure. By starting with this model, the master's or doctoral candidate establishes, in advance, all steps and activities in the research project schedule, which is – or should be – the starting point for any master's or doctorate. Victoria University (2020)Victoria University. (2020). Guidelines on thesis by publication at Victoria University. Retrieved in 2020, September 17, from https://www.vu.edu.au/sites/default/files/research/pdfs/Guidelines-on-thesis-by-publication.pdf
https://www.vu.edu.au/sites/default/file...
guidelines points out that the project gates and milestones must reflect how planned work and publication outputs will enable the thesis to be completed within the standard doctoral degree period. Additionally, the University of Reading (2016)University of Reading. (2016, November 02). Writing a thesis as a collection of papers. Retrieved in 2020, September 17, from https://www.reading.ac.uk/web/files/graduateschool/gradschwritingthesisascollectionofpapers.pdf
https://www.reading.ac.uk/web/files/grad...
draws that this structure may not be suitable for projects where publishable results come near the end of the three years.

Nevertheless, the scenario might change during the candidate's course, which may demand modification in thesis format. It may occur because the candidate did not or will not achieve the planned schedule of producing publications and getting them accepted for publication, so one can judge more appropriate to produce a thesis in the standard format (Victoria University, 2020Victoria University. (2020). Guidelines on thesis by publication at Victoria University. Retrieved in 2020, September 17, from https://www.vu.edu.au/sites/default/files/research/pdfs/Guidelines-on-thesis-by-publication.pdf
https://www.vu.edu.au/sites/default/file...
). Alternatively, the same reference states that it may be that many papers have been written and accepted for publication in the course of the candidature to date, and the supervisor/s and candidate consider that it would be advantageous for the thesis submission to be able to directly incorporate these by changing from the standard to the paper-based thesis. For instance, suppose a doctorate student that initially planned his/her thesis in the traditional format, because the graduate program where he/she studies does not permit PBT. Afterwards, the graduate program takes a new decision that allows PBT. Then, the student may choose, in agreement with his/her supervisor, to change to the PBT structure. Thus, some document features might change. This could bring about benefits such as: faster writing of the thesis (Breimer & Mikhailidis, 1993Breimer, L. H., & Mikhailidis, D. P. (1993). Towards a doctoral thesis through published works. Biomedicine and Pharmacotherapy, 47(9), 403-407. http://dx.doi.org/10.1016/0753-3322(93)90106-U. PMid:8068863.
http://dx.doi.org/10.1016/0753-3322(93)9...
; University of Reading, 2016University of Reading. (2016, November 02). Writing a thesis as a collection of papers. Retrieved in 2020, September 17, from https://www.reading.ac.uk/web/files/graduateschool/gradschwritingthesisascollectionofpapers.pdf
https://www.reading.ac.uk/web/files/grad...
), more opportunity to enhance writing and selecting journal, and responding to reviewers’ comments skills (University of Reading, 2016University of Reading. (2016, November 02). Writing a thesis as a collection of papers. Retrieved in 2020, September 17, from https://www.reading.ac.uk/web/files/graduateschool/gradschwritingthesisascollectionofpapers.pdf
https://www.reading.ac.uk/web/files/grad...
), and potential future visibility of the research work.

On the other hand, difficulties may arise. The structure change during the doctorate demands a better connection among the articles. Although the papers might have some interrelationship, it had to be better stitched, since the research student did not foresee his/her thesis by papers development in the early stages of the doctoral research. Moreover, it is necessary to work with more emphasis on the research methodological articulation among the articles: objectives, research problem, and, more generally, the discussion, contributions, and conclusions of the thesis as a whole. This effort is needed because structuring and carrying out the thesis as a collection of articles is not about merely binding the papers and writing an introduction and conclusions sections, as one may think. The final paper-based thesis document must be a cohesive, coherent, integrated, and robust piece of work, where each paper contributes to the overall thesis, as recommended by the University of Reading (2016)University of Reading. (2016, November 02). Writing a thesis as a collection of papers. Retrieved in 2020, September 17, from https://www.reading.ac.uk/web/files/graduateschool/gradschwritingthesisascollectionofpapers.pdf
https://www.reading.ac.uk/web/files/grad...
, the Queensland University of Technology (2014)Queensland University of Technology. (2014, October). Thesis by publication candidate & supervisor guidelines. Retrieved in 2020, September 17, from https://cms.qut.edu.au/__data/assets/pdf_file/0004/451903/faculty-of-education-thesis-by-publication-guidelines.pdf
https://cms.qut.edu.au/__data/assets/pdf...
, the Western Sydney University (2016)Western Sydney University – WSU. (2016). Higher degree research: guidelines for examiners 2016. Retrieved in 2020, September 17, from https://www.westernsydney.edu.au/__data/assets/pdf_file/0019/1093024/GRSC1139_HDR_Guidelines_for_Examiners.pdf
https://www.westernsydney.edu.au/__data/...
, and the Graduate Program of Production Engineering from the Federal University of Santa Catarina (PPGEP-UFSC, 2018Universidade Federal de Santa Catarina – UFSC. Programa de Pós-graduação em Engenharia de Produção – PPGEP. (2018, November 7). Resolução 001/PPGEP/2018, de 07/11/2018. Boletim Oficial da UFSC. Retrieved in 2020, September 17, from https://treinamanto5.paginas.ufsc.br/files/2011/07/Resolucao-002_2018_Dissertacao-Tese_FormatoArtigos.pdf
https://treinamanto5.paginas.ufsc.br/fil...
).

3.2. The Introduction chapter

In general, the introductory chapter is relatively similar in a traditional and paper-based thesis (Abdolmalaki et al., 2018Abdolmalaki, S. G., Tan, H., Abdullah, A. N. B., Sharmini, S., & Imm, L. G. (2018). Introduction chapter of traditional and article-based thess: a comparison of rhetorical structures and linguistic realisations. Journal of Language Studies, 19(1), 116-135. http://doi.org/10.17576/gema-2019-1901-07.
http://doi.org/10.17576/gema-2019-1901-0...
). A synopsis of each research paper and claim for the novelty of research/paper was identified as steps for the PBT’s introduction. The authors argue that the reason for those inclusions was that a PBT needs to draw short summaries of each article and lay claims to the novelty of the mentioned manuscripts. Additionally, “[...] providing insights into each paper that have been compiled in the thesis and laying claims to the uniqueness of the research demonstrate the scholarship of the article-based thesis writers” (Abdolmalaki et al., 2018Abdolmalaki, S. G., Tan, H., Abdullah, A. N. B., Sharmini, S., & Imm, L. G. (2018). Introduction chapter of traditional and article-based thess: a comparison of rhetorical structures and linguistic realisations. Journal of Language Studies, 19(1), 116-135. http://doi.org/10.17576/gema-2019-1901-07.
http://doi.org/10.17576/gema-2019-1901-0...
, p. 120). Moreover, the University of Reading (2016)University of Reading. (2016, November 02). Writing a thesis as a collection of papers. Retrieved in 2020, September 17, from https://www.reading.ac.uk/web/files/graduateschool/gradschwritingthesisascollectionofpapers.pdf
https://www.reading.ac.uk/web/files/grad...
, Faculty of Nursing – University of Alberta (2015)Faculty of Nursing – University of Alberta (2015). Guidelines for a Paper Based Thesis. Edmonton: University of Alberta. Retrieved in 2021, May 03rd, from https://www.ualberta.ca/nursing/media-library/forms/grad/guidelines-paper-based-thesis.pdf.
https://www.ualberta.ca/nursing/media-li...
, and the University of New England (2015)University of New England. (2015). HDR: Higher Degree Research thesis by publication guideline. Retrieved in 2020, September 17, from https://policies.une.edu.au/view.current.php?id=00284
https://policies.une.edu.au/view.current...
outline the importance of showing the link among the papers, as well as the research questions they address to comply with the thesis as a whole.

The introduction of the paper-based thesis, similar to the traditional model, must present the research problem, objectives (general and specific), and the relevance of the work. However, there is a difference here. In a PBT, there is a need for a robust connection between the research problem/opportunity, and the objectives and contents of the final document articles, reminding that an article is a consequence of a research development/execution, and should not be a purpose itself (Cauchick-Miguel et al., 2017Cauchick-Miguel, P. A., Campos, L. M. S., Jabbour, C. J. C., & Jabbour, A. B. L. S. (2017). Elaboração de artigos acadêmicos: estruturas, métodos e técnicas. Rio de Janeiro: Elsevier.). An alternative to establishing this more robustly is to align one or more specific goals to the objectives stated in each article that build up the whole research so that they can meet, entirely, the general purpose of the study/thesis. For aligning it consistently, this study suggests emphasizing the content of the thesis itself instead of the content of the articles, so that the reader can clearly understand the final document as a whole, whether he/she is the board examiner, or a future researcher interested in the work. Additionally, although the theoretical basis could be outlined in the introduction, this study suggests the possibility for the researcher to publish theoretical-based articles (e.g., systematic literature reviews) to compose the theoretical basis of the PBT.

3.3. The Results and discussion chapter(s)

For the chapters of research results (which are usually represented by the articles), one can avoid an ipsis litteris reproduction of the article structure. Some of these chapters may take the form of blind peer-reviewed papers (Queensland University of Technology, 2014Queensland University of Technology. (2014, October). Thesis by publication candidate & supervisor guidelines. Retrieved in 2020, September 17, from https://cms.qut.edu.au/__data/assets/pdf_file/0004/451903/faculty-of-education-thesis-by-publication-guidelines.pdf
https://cms.qut.edu.au/__data/assets/pdf...
). In fact, at the beginning of these chapters, it is understood that there is no need to present the publications abstract precisely. However, this study suggests inserting an initial informative paragraph, which summarizes each chapter and links them with the structure of the thesis, in the sense of designing the research problem (and/or research opportunity), general and specific objectives, and the role of the chapters in the thesis structure. Short bridging chapters may be useful to link those publications together (Queensland University of Technology, 2014Queensland University of Technology. (2014, October). Thesis by publication candidate & supervisor guidelines. Retrieved in 2020, September 17, from https://cms.qut.edu.au/__data/assets/pdf_file/0004/451903/faculty-of-education-thesis-by-publication-guidelines.pdf
https://cms.qut.edu.au/__data/assets/pdf...
). Thus, it is necessary to build a text with unambiguous arguments, i.e., not just a formality or a complete copy of the article's abstract. Therefore, the present work recommends that this brief chapter’s presentation does not simply consist of a summary, but the presentation of the contents that will be presented, with emphasis in the contribution of the chapter linked to the research phases or stages, previously outlined in the research design and methods.

Another critical point involves the discussion of the results. Although some graduate programs do not require a specific chapter for general discussions and contributions of the thesis, the examining board may, after the thesis defense, recommend it, aiming at concatenating the analysis of the thesis' results and outcomes, connecting the partial results of each article. Some institutions demand this chapter when the final research document has the articles collection format, as shown in the following excerpt taken from the Manual of Dissertations and Thesis of the Federal University of Santa Maria (Universidade Federal de Santa Maria, 2015Universidade Federal de Santa Maria – UFSM. (2015). Manual de dissertações e teses da UFSM: estrutura e apresentação. Santa Maria: Editora UFSM. Retrieved in 2020, September 17, from https://www.ufsm.br/orgaos-suplementares/biblioteca/wp-content/uploads/sites/362/2019/01/Manual_de_Dissertacoes_e_Teses-2015-3.pdf
https://www.ufsm.br/orgaos-suplementares...
, p. 49):

For monographies that contain at least two articles, a chapter, called discussion, will be required in order to integrate and discuss the theme and results of the articles as a whole. This chapter should come right after the articles and before the conclusions. (authors’ translation from the original document in Portuguese).

In the chapters with articles, another critical issue is to avoid repetition when presenting an overview of the findings and their respective discussions and implications (Faculty of Nursing – University of Alberta, 2015Faculty of Nursing – University of Alberta (2015). Guidelines for a Paper Based Thesis. Edmonton: University of Alberta. Retrieved in 2021, May 03rd, from https://www.ualberta.ca/nursing/media-library/forms/grad/guidelines-paper-based-thesis.pdf.
https://www.ualberta.ca/nursing/media-li...
). Thus, the doctoral (or master's) candidate must synthesize the significance of the results, in order to facilitate the examination process (University of New England, 2015University of New England. (2015). HDR: Higher Degree Research thesis by publication guideline. Retrieved in 2020, September 17, from https://policies.une.edu.au/view.current.php?id=00284
https://policies.une.edu.au/view.current...
; University of Western Australia, 2020University of Western Australia. (2020). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/students/thesis/series
http://www.postgraduate.uwa.edu.au/stude...
).

3.4. The conclusions chapter

About the conclusions of a paper-based thesis, similarly to the traditional model, these should present the main concluding remarks from the discussion, as well as the theoretical and empirical (if existed) implications, research limitations, and opportunities for future work due to research results. The present work recommends a presentation of an overview of the main findings so that the contents already written in the articles does not appear again, avoiding repetition. Similar decision should be taken when pointing out the limitations of the thesis and the opportunities for future work. As mentioned by the University of New England (2015)University of New England. (2015). HDR: Higher Degree Research thesis by publication guideline. Retrieved in 2020, September 17, from https://policies.une.edu.au/view.current.php?id=00284
https://policies.une.edu.au/view.current...
, University of Western Australia (2020)University of Western Australia. (2020). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/students/thesis/series
http://www.postgraduate.uwa.edu.au/stude...
and Universidade Federal de Santa Maria (2015Universidade Federal de Santa Maria – UFSM. (2015). Manual de dissertações e teses da UFSM: estrutura e apresentação. Santa Maria: Editora UFSM. Retrieved in 2020, September 17, from https://www.ufsm.br/orgaos-suplementares/biblioteca/wp-content/uploads/sites/362/2019/01/Manual_de_Dissertacoes_e_Teses-2015-3.pdf
https://www.ufsm.br/orgaos-suplementares...
, p. 50) guidelines:

A general conclusion outlines how the research manuscripts link together and as a collective address the philosophy of the research as well as highlighting the original contribution of the body of knowledge in the chosen area. This chapter would normally also include recommendations for further work. References for the Conclusion are included at the end of this chapter. (University of New England, 2015University of New England. (2015). HDR: Higher Degree Research thesis by publication guideline. Retrieved in 2020, September 17, from https://policies.une.edu.au/view.current.php?id=00284
https://policies.une.edu.au/view.current...
).

The concluding chapter needs to draw together the main findings of the thesis and synthesise the significance of the results. (University of Western Australia, 2020University of Western Australia. (2020). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/students/thesis/series
http://www.postgraduate.uwa.edu.au/stude...
).

When dissertations or theses contain two or more articles, there must be a general conclusion. (Universidade Federal de Santa Maria, 2015Universidade Federal de Santa Maria – UFSM. (2015). Manual de dissertações e teses da UFSM: estrutura e apresentação. Santa Maria: Editora UFSM. Retrieved in 2020, September 17, from https://www.ufsm.br/orgaos-suplementares/biblioteca/wp-content/uploads/sites/362/2019/01/Manual_de_Dissertacoes_e_Teses-2015-3.pdf
https://www.ufsm.br/orgaos-suplementares...
, p. 50; authors’ translation from the original document in Portuguese).

Furthermore, one can highlight that a potential contribution of a paper-based thesis is the alignment and stitching of these works together to bring what each discovery or limitation brought inspiration for the development of later articles. It demonstrates maturity and robustness, as well as a logical and orderly sequence of the specific contributions of each research paper. This construction is essential to explain the relationship between the publications (articles) and the thesis. The fact that the research work has published/accepted articles does not assure, nor does it replace, the authority and responsibility of the board in evaluating the final document, even if these articles are accepted/published in high impact journals (Queensland University of Technology, 2014Queensland University of Technology. (2014, October). Thesis by publication candidate & supervisor guidelines. Retrieved in 2020, September 17, from https://cms.qut.edu.au/__data/assets/pdf_file/0004/451903/faculty-of-education-thesis-by-publication-guidelines.pdf
https://cms.qut.edu.au/__data/assets/pdf...
; Western Sydney University, 2016Western Sydney University – WSU. (2016). Higher degree research: guidelines for examiners 2016. Retrieved in 2020, September 17, from https://www.westernsydney.edu.au/__data/assets/pdf_file/0019/1093024/GRSC1139_HDR_Guidelines_for_Examiners.pdf
https://www.westernsydney.edu.au/__data/...
; University of Western Australia, 2020University of Western Australia. (2020). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/students/thesis/series
http://www.postgraduate.uwa.edu.au/stude...
). The papers correspond to the partial results resulting from broader research work and which, separately, should not be considered a thesis as a whole, but part of it.

3.5. The research design and methods chapter

The research design chapter on a paper-based thesis is significantly critical, which is why this article has separated a specific section regarding this thesis by article chapters. Research work, whether at the master's or doctoral level, is unlikely to be successful if the research methods are not well-designed and properly justified, as well as without sufficiently robust details to allow replication. Examiners normally (if not always) will assess the candidate’s work “[...] to determine if the methods adopted are appropriate to the subject matter and are appropriately applied” (Western Sydney University, 2016Western Sydney University – WSU. (2016). Higher degree research: guidelines for examiners 2016. Retrieved in 2020, September 17, from https://www.westernsydney.edu.au/__data/assets/pdf_file/0019/1093024/GRSC1139_HDR_Guidelines_for_Examiners.pdf
https://www.westernsydney.edu.au/__data/...
, p. 9).

The first important aspect to highlight is that it is necessary to offer a logical sequence of the methodological procedures applied in the research so that there is no disconnection between these procedures. The author of a PBT must be thorough with the research design chapter because it is necessary to connect and align the methodological procedures applied in each of the articles. When well developed, it draws a clear connection between the methodological stages of the work as a whole.

Thus, it is also necessary to detail the thesis’ contents that are not possible to incorporate in the papers, especially considering that the articles have length limitations (number of pages or, more usual, number of words), which the Curtin University (2019)Curtin University. (2019). Guidelines for thesis by publication and hybrid thesis. Retrieved in 2020, September 17, from https://students.curtin.edu.au/wp-content/uploads/sites/6/2019/08/Guidelines-for-Thesis-by-Publication-and-Hybrid-Thesis-15.8.19.pdf
https://students.curtin.edu.au/wp-conten...
recommends in their guidelines: one chapter in the final document that summarizes the thesis or dissertation research methods. Therefore, this recommendation becomes crucial. To address that, it is possible to use structured tables that synthesize the unfolding of all thesis (e.g., selected keywords, databases used, coding used in content analysis, interview data, evidence sources, etc.), so complementing the discussion of these tables in the text body itself. For some thesis themes, the research methods chapter might be represented by a methodological paper, which may consist, for instance, of: (i) a mathematical model developed by the candidate, which would be further tested/simulates, or (ii) a conceptual framework that would be empirically tested later.

Another possibility that might help to structure the thesis research methods is to subdivide this chapter into ‘modules’ or ‘blocks’. This may facilitate the synthesis of the steps of the applied methodological procedures. For instance, in a thesis involving field research, one can subdivide as follows:

  1. i

    Phase 1 – Literature review: part 1 (bibliometric/descriptive analysis);

  2. ii

    Phase 2 – Literature review: part 2 (content analysis and conceptual framework development);

  3. iii

    Phase 3 – Empirical research – exploratory stage;

  4. iv

    Phase 4 – Empirical investigation – confirmatory stage.

By using the previous subdivision, one can better detail each stage of the research. However, it is noteworthy mentioning that the above subdivision is not unique and is up to the research author to decide a logical structure that facilitates the report of his/her research methods. In other situations, it is possible to substitute the research methods chapter by a paper that embeds the referred content (Queensland University of Technology, 2014Queensland University of Technology. (2014, October). Thesis by publication candidate & supervisor guidelines. Retrieved in 2020, September 17, from https://cms.qut.edu.au/__data/assets/pdf_file/0004/451903/faculty-of-education-thesis-by-publication-guidelines.pdf
https://cms.qut.edu.au/__data/assets/pdf...
), as discussed earlier. In addition, in the development of a conceptual framework or a reference model, the candidate may publish an article that presents its construction, by detailing all criteria and literature examined to build the model, to further apply it empirically.

3.6. Authorship and co-authorship

One of the existing discussions about paper-based thesis involves co-authorship, especially concerning the research advisor/supervisor. When examining universities' regulations and guidelines for PBT, the authors found out that authorship and co-authorship rules were one of the most common issues discussed in all those documents.

Considering articles are usually authored by the doctoral or master's student and his supervisor as a co-author, one can believe that the thesis was, in significant part, developed jointly by the student and the advisor. It is not the purpose of a doctorate title. Even though, more often the student has predominantly developed the research and its articles in co-creation with the advisor acting with less participation. In this sense, some institutions have proposed some guidelines to minimize the concerns from this situation. The next excerpt shows an example (bold highlights made by the authors):

- At least one of the papers must be a single authored.

- In the case of co-authored papers, the doctoral candidate will submit a separate note about the nature and extent of his personal contribution to the paper(s), which will be signed by the co-authors. The PhD candidate’s own contribution to co-authored papers shall be substantial. (Graduate Institute of Geneva, 2016Graduate Institute of Geneva. (2016). Doctoral programme: guidelines for paper-based theses. Retrieved in 2020, September 17, from https://graduateinstitute.ch//sites/default/files/2019-03/Guidelines_Paper-based_thesis_approved.pdf?_ga=2.224505118.508123277.590091744-2119315886.1590091744
https://graduateinstitute.ch//sites/defa...
, p. 1).

The previous excerpt showed the main requirements to accept the PBT. The condition is that the candidate has had the predominant role in conducting the research. They also require it through another rule. The student must be the sole author of at least one of the three articles (which should be accepted or potentially accepted for publication) that will be part of the final document (Graduate Institute of Geneva, 2016Graduate Institute of Geneva. (2016). Doctoral programme: guidelines for paper-based theses. Retrieved in 2020, September 17, from https://graduateinstitute.ch//sites/default/files/2019-03/Guidelines_Paper-based_thesis_approved.pdf?_ga=2.224505118.508123277.590091744-2119315886.1590091744
https://graduateinstitute.ch//sites/defa...
, p. 1). In this way, it would be possible to mitigate, at least partially, the potential problem of the co-authored thesis between student and advisor. Other institutions also deal with that issue, such as the following excerpts point out (bold highlights made by the authors):

For each paper a declaration about authorship and contribution to the work, including acknowledgement of joint or multiple authors and verification of each of their contributions and permissions. (Victoria University, 2020Victoria University. (2020). Guidelines on thesis by publication at Victoria University. Retrieved in 2020, September 17, from https://www.vu.edu.au/sites/default/files/research/pdfs/Guidelines-on-thesis-by-publication.pdf
https://www.vu.edu.au/sites/default/file...
).

Publications submitted as part of the Doctoral Thesis by Publication can be either sole or co-authored. However, where papers are co-authored, the candidate must:

• be the lead author; and

• have contributed at least 50% to the publication. (Queensland University of Technology, 2014Queensland University of Technology. (2014, October). Thesis by publication candidate & supervisor guidelines. Retrieved in 2020, September 17, from https://cms.qut.edu.au/__data/assets/pdf_file/0004/451903/faculty-of-education-thesis-by-publication-guidelines.pdf
https://cms.qut.edu.au/__data/assets/pdf...
, p. 2).

Acquisition of funding, collection of data, or general supervision alone does not constitute authorship. Each author will need to submit a signed Statement of Contribution of Co-Authors for Thesis by Published Papers and disclose their percentage of contribution on submission of the thesis. Students are advised to manage these processes along the way, and to ensure adequate records are kept. (Queensland University of Technology, 2014Queensland University of Technology. (2014, October). Thesis by publication candidate & supervisor guidelines. Retrieved in 2020, September 17, from https://cms.qut.edu.au/__data/assets/pdf_file/0004/451903/faculty-of-education-thesis-by-publication-guidelines.pdf
https://cms.qut.edu.au/__data/assets/pdf...
, p. 2).

To be eligible for co-authorship, each author must have made a significant contribution to all three stages in the development of the paper, including:

(i) the conception and design of the project

(ii) analysis and interpretation of research data; and

(iii) drafting significant parts of the paper or critically revising it so as to contribute to the interpretation. (Queensland University of Technology, 2014Queensland University of Technology. (2014, October). Thesis by publication candidate & supervisor guidelines. Retrieved in 2020, September 17, from https://cms.qut.edu.au/__data/assets/pdf_file/0004/451903/faculty-of-education-thesis-by-publication-guidelines.pdf
https://cms.qut.edu.au/__data/assets/pdf...
, p. 2).

The student should normally be the lead author of the paper, and certainly a significant author in that they must have made a substantial contribution to any jointly authored paper. The student should include in the thesis a statement outlining his or her specific contribution to any jointly authored paper that is included, indicating what components of the work were carried out by the candidate and what components were carried out by other authors, and indicating the estimated percentage contribution made by the candidate. This statement should be approved by the supervisor before being included in the thesis. (University of Reading, 2016University of Reading. (2016, November 02). Writing a thesis as a collection of papers. Retrieved in 2020, September 17, from https://www.reading.ac.uk/web/files/graduateschool/gradschwritingthesisascollectionofpapers.pdf
https://www.reading.ac.uk/web/files/grad...
, p. 2-3).

Where there are multiple authors, the candidate will normally be the first author. (Western Sydney University, 2016Western Sydney University – WSU. (2016). Higher degree research: guidelines for examiners 2016. Retrieved in 2020, September 17, from https://www.westernsydney.edu.au/__data/assets/pdf_file/0019/1093024/GRSC1139_HDR_Guidelines_for_Examiners.pdf
https://www.westernsydney.edu.au/__data/...
, p. 8).

It is expected that you will be the major contributor to any papers that you include within your thesis, and this commonly means you will be the first author. (University of Western Australia, 2020University of Western Australia. (2020). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/students/thesis/series
http://www.postgraduate.uwa.edu.au/stude...
).

If you include co-authored papers in your thesis, it is critical that your contribution to the paper is made clear to the examiner. As they are examining your work and not the work of your co-authors there needs to be clear delineation of the work that needs to be examined. (University of Western Australia, 2020University of Western Australia. (2020). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/students/thesis/series
http://www.postgraduate.uwa.edu.au/stude...
).

From all those previous excerpts, one can notice that the common worry about authorship and co-authorship involves the real contribution of the candidate in the papers that he/she will use in his/her thesis. Most courses demand the candidate to be the first or sole author of the thesis’ manuscripts, as well as a document where the student declares his/her participation in the articles, pointing out the level of involvement in the various development stages. Based on this evidence, this study recommends that most of the papers must have the candidate as the first author, which indicates that he/she is the leading researcher.

3.7. Copyrights

Copyright is another frequent topic in all analyzed documents. Since manuscripts involve copyright issues regarding publishers, journals, and the candidate's university, it is crucial to deal appropriately with this aspect. Regarding the papers' copyright, this study points out that it is essential to recover and present them in the final document. One suggestion is to include them at the end of the text in an appendix section. Some universities and graduate programs foresee and require this in their resolutions, which is the sole responsibility of the master's or doctoral student. Additionally, the permission of co-authors (when pertinent) is also required so that the referred work can compose the student's thesis.

Many institutions have a specific topic covering copyright permission for the PBT model, as can be seen in the following excerpts (bold highlights made by the authors):

If you include published papers within your thesis, at the time the thesis is published in the UWA Research Repository, consideration will need to be given to the copyright of the papers in the thesis. Prior to choosing to publish in a journal, you may wish to consider the publishers copyright policy to determine if you can include a copy of a published paper in the final version of your thesis that is available to the public. (University of Western Australia, 2020University of Western Australia. (2020). Thesis as a series of papers. Retrieved in 2020, September 17, from http://www.postgraduate.uwa.edu.au/students/thesis/series
http://www.postgraduate.uwa.edu.au/stude...
).

A statement must be included in the front matter of the thesis to indicate that permission regarding copyright has been obtained from publishers, where necessary. (Curtin University, 2019Curtin University. (2019). Guidelines for thesis by publication and hybrid thesis. Retrieved in 2020, September 17, from https://students.curtin.edu.au/wp-content/uploads/sites/6/2019/08/Guidelines-for-Thesis-by-Publication-and-Hybrid-Thesis-15.8.19.pdf
https://students.curtin.edu.au/wp-conten...
, p. 2).

The candidate must check with each Publisher whether or not there are any restrictions regarding copyright, format and style before the inclusion of the paper in the thesis. When papers are published the copyright is usually assigned to the journal. Therefore the candidate must take steps to avoid copyright infringement. Laws may differ from country to country. Therefore, it is important to check with each publisher. This should be done at the time the paper is accepted for publication. (Curtin University, 2019Curtin University. (2019). Guidelines for thesis by publication and hybrid thesis. Retrieved in 2020, September 17, from https://students.curtin.edu.au/wp-content/uploads/sites/6/2019/08/Guidelines-for-Thesis-by-Publication-and-Hybrid-Thesis-15.8.19.pdf
https://students.curtin.edu.au/wp-conten...
, p. 3).

As pointed out in the latter excerpt, in some cases paper-based thesis guidelines may even indicate the moment to check for copyright permission with the publisher. Usually, publishers do not set restrictions on authors to publish articles in theses. Some of them even encourage authors to do that. This work identified that Taylor & Francis (2020)Taylor & Francis. (2020). Understanding copyright for journal authors. Retrieved in 2020, September 17, from https://authorservices.taylorandfrancis.com/publishing-your-research/moving-through-production/copyright-for-journal-authors/
https://authorservices.taylorandfrancis....
has a specific restriction about the ‘version of record’. Table 2 presents some publishers and their positions regarding papers in PBT.

Table 2
Publishers’ position about articles copyright (for paper-based thesis).

3.8. Synthesis of the literature and guidelines on paper-based thesis model

After searching for resolutions and manuscripts related to the compound term ‘thesis by articles’, the authors found out some relevant points associated with the structuring of a paper-based thesis. This is still a relatively nascent practice concerning thesis (and dissertations) with this structure within the Brazilian engineering courses context. It usually requires both more effective and robust recommendations for research guidelines, as well as more significant discussion about the benefits and disadvantages of structuring theses by publications. On the other hand, some graduate programs in the country already have resolutions defining guidelines for such structuring, although they could be slightly more detailed in some requirements. The authors understand that the guidelines from overseas universities are more enlightening in some items, such as copyright permission, authorship, and co-authorship, and about the contents demanded in each thesis chapter.

Moreover, this work recommends that PBT should have articles that build a clear and robust connection along with each other. It must have an introduction and conclusion that, among other requirements, should place the papers in a broad context of the literature, elaborating (when appropriate) a theoretical, conceptual, and methodological construction about the articles. It can also highlight the alignment between them, aiming to discuss possibilities for future research. Thus, it is also necessary to carefully select the papers that will be part of the final document, avoiding criticisms such as the lack of internal consistency and the lack of coherence and cohesion in the thesis article portfolio (Badley, 2009Badley, G. (2009). Publish and be doctor-rated: the PhD by published work. Quality Assurance in Education: an International Perspective, 17(4), 331-342. http://dx.doi.org/10.1108/09684880910992313.
http://dx.doi.org/10.1108/09684880910992...
; Sharmini et al., 2015Sharmini, S., Spronken-Smith, R., Golding, C., & Harland, T. (2015). Assessing the doctoral thesis when it includes published work. Assessment & Evaluation in Higher Education, 40(1), 89-102. http://dx.doi.org/10.1080/02602938.2014.888535.
http://dx.doi.org/10.1080/02602938.2014....
; Peacock, 2017Peacock, S. (2017). The PhD by publication. International Journal of Doctoral Studies, 12, 123-135. http://dx.doi.org/10.28945/3781.
http://dx.doi.org/10.28945/3781...
).

Furthermore, candidates must not include works from previous master’s or doctoral research as part of the thesis being developed by the student who authored the work (Graduate Institute of Geneva, 2016Graduate Institute of Geneva. (2016). Doctoral programme: guidelines for paper-based theses. Retrieved in 2020, September 17, from https://graduateinstitute.ch//sites/default/files/2019-03/Guidelines_Paper-based_thesis_approved.pdf?_ga=2.224505118.508123277.590091744-2119315886.1590091744
https://graduateinstitute.ch//sites/defa...
). In the examined Brazilian guidelines, institutions do not usually allow previous work to be incorporated in the PBT final document.

Lastly, different universities indeed have different guidelines on thesis writing (Nguyen & Pramoolsook, 2014Nguyen, L. T. T., & Pramoolsook, I. (2014). Rhetorical structure of Introduction chapters written by novice Vietnamese TESOL postgraduates. 3L: Language, Linguistics, Literature , 20(1), 61-74. http://dx.doi.org/10.17576/3L-2014-2001-05.
http://dx.doi.org/10.17576/3L-2014-2001-...
). Nevertheless, this paper defends that, still, there are common features that candidates, supervisors, and examiners need to consider when conducting, supervising, or examining a PBT. Table 3 provides a summary of diverse recommendations for the development of a paper-based thesis, developed from the analysis of several documents identified in this work. In the first column, this study establishes some fundamental requirements, while in the following columns it draws the approach suggestions according to each requirement. Critical subjects regarding paper-based thesis were presented here. The next section draws some concluding remarks from those analyses.

Table 3
List of recommendations for the development of paper-based thesis.

4. Conclusions

This work discussed the most critical points to develop a paper-based thesis, aiming at achieving quality and robustness in the construction of research works with this format. It also deals with characteristics and possible advantages and disadvantages when developing a paper-based thesis. Thus, the paper has as the main contribution proposals for systematic development, and recommendations for developing the PBT format, from the initial decision making to the research project ending, considering the possible restrictions, as well as its potential and benefits.

4.1. Theoretical and practical implications

The objective of this study was to provide recommendations for paper-based thesis, based on the analysis of research papers related to the PBT subject and documentation (internal regulations) identified in graduate programs in Brazilian and overseas universities. The development of PBT is not necessarily more straightforward than the traditional model, since there are several challenges involved. Some of those are copyrights of the accepted/published papers, co-authorship, the connection among articles, and the researcher's role (thesis or dissertation author) when conducting his/her research work.

When comparing the early decision for the paper-based model, with the model's transition during the master's or doctoral degree, this study recommends beginning with the PBT structure since the early months of the endeavor. Hence, this study recommends taking this decision as soon as possible, as it increases the possibility of more effective research project's planning and execution, reducing the mentioned difficulties previously in conducting research in the context of the paper-based format. Finally, it is crucial to emphasize that the paper-based thesis is not just about compiling a series of articles, binding them, and delivering them together with an introduction and conclusions. When adopting this model, the doctoral/master's candidate must build a logical, cohesive, coherent, and robust connection between the articles that will be part of the final document, in order to characterize it as a master's dissertation or a doctoral thesis. The authors also point out that the most crucial aspect is the proper conduction of the whole research (both the candidate and the supervisor). Thus, the paper-based thesis planning must foresee how each paper will contribute to the thesis, avoiding academic paralysis for the doctoral/master’s candidate. It is also worth strengthening that there is a possibility of publishing theoretical-conceptual articles like a robust literature review paper in order to build the PBT theoretical background.

4.2. Limitations and further research opportunities

This study had an emphasis on the internal regulations of graduate programs, being this the main limitation of the paper. Further research could analyze in-depth, research papers dealing with paper-based thesis, as well as dissertations and thesis developed with the paper-based format, according to the areas of knowledge (e.g., engineering, business, etc.). This research approach would be beneficial to identify the specific issues of each area and to analyze how these peculiarities may affect the development of PBT, both in Brazil and overseas.

Acknowledgements

The authors do appreciate the recommendations of the reviewers and the editor who dedicated their time and provided remarkable suggestions and recommendations that enhanced our manuscript.

  • How to cite this article: Kubota, F. I., Cauchick-Miguel, P. A., Tortorella, G., & Amorim, M. (2021). Paper-based thesis and dissertations: analysis of fundamental characteristics for achieving a robust structure. Production, 31, e20200100. https://doi.org/10.1590/0103-6513.20200100

References

Publication Dates

  • Publication in this collection
    09 June 2021
  • Date of issue
    2021

History

  • Received
    17 Sept 2020
  • Accepted
    26 Apr 2021
Associação Brasileira de Engenharia de Produção Av. Prof. Almeida Prado, Travessa 2, 128 - 2º andar - Room 231, 05508-900 São Paulo - SP - São Paulo - SP - Brazil
E-mail: production@editoracubo.com.br