The design of the certification service of quality management systems ISO 9001: a case study in a military organization of the Brazilian Navy

O projeto de concepção do serviço de certificação de sistemas de gestão da qualidade ISO 9001: um estudo de caso em uma organização militar da Marinha do Brasil

Abstract:

The Brazilian Navy is preparing to design, build and repair nuclear submarine. One of the initiatives adopted was the creation of the Naval Agency for Nuclear Safety and Quality (AgNSNQ) which aims to assist the licensing of nuclear and naval facilities and conformity assessment of Quality Management Systems. The concept of the certification service of management systems needs to be evaluated, before expending resources and people to develop. This article aimed to assess the characteristics of AgNSNQ to design the ISO 9001 QMS certification service through the service project model. A case study was conducted to evaluate the characteristics of the AgNSNQ concerning the criteria. The article's contributions indicate that AgNSNQ should primarily promote efforts to train its auditors to create favorable conditions for designing the ISO 9001 QMS certification service.

Keywords:
Certification; ISO 9001; Quality Management Systems; Service Design Project

Resumo:

A Marinha do Brasil se prepara para projetar, construir e manter seu primeiro submarino nuclear, uma das iniciativas adotadas foi a criação da Agência Naval de Segurança Nuclear e Qualidade (AgNSNQ) que terá como propósito contribuir para o licenciamento de meios e instalações navais nucleares e avaliar a conformidade de sistemas de gestão, produtos e serviços, contudo o serviço de certificação que a AgNSNQ pretende prover precisa ser avaliado antes que se empregue pessoas e recursos para o seu desenvolvimento. Este artigo tem como objetivo avaliar as características da AgNSNQ para conceber o serviço de certificação de SGQ ISO 9001, através do grau de atendimento à etapa de Projeto de Concepção de Serviço do modelo de Desenvolvimento de Serviços. Um estudo de caso foi realizado para a avaliação das características da AgNSNQ em confronto aos critérios. As contribuições do artigo indicam que a AgNSNQ deve promover principalmente esforços em capacitação de seus auditores para criar condições favoráveis para conceber o serviço de certificação de SGQ ISO 9001.

Palavras-chave:
Certificação; ISO 9001; Sistemas de Gestão da Qualidade; Projeto de Concepção de Serviço

1 Introduction

In the late 1950s, the escalating military power between the United States of America and the Union of Soviet Socialist Republics unleashed a nuclear and space race involving complex systems with high levels of reliability. Quality was consolidated as a vital tool for employment in the nuclear area and used after by US Navy Nuclear Program (submarines and aircraft carriers) under the name of Quality Assurance (Fernandes, 2011).

In 1959, the US developed the MIL-Q-9858A (Quality Program Requirements) and MIL-I-45208A (Inspection System Requirements) standard, the first quality standards for military procurement. Already in 1968, the North Atlantic Treaty Organization (NATO) adopted the Allied Quality Assurance Procedures (AQAP), also used for military purchases. These standards, together with the British Standard BSI-5750 elaborated in 1979, served as a basis for the creation of ISO 9000 in 1987 and ISO 9001 in 1994) (Seddon, 2000).

As of December 31, 2016, there were 1,106,356 valid ISO 9001 certificates worldwide, an increase of 7% relative to 2015; this tool is the most used by organizations from developed and developing countries to meet the needs and expectations of its customers (ISO, 2017a).

Meeting the requirements of ISO 9001 can be certified by an independent organization, this voluntary process proves that the organization has a Quality Management System (QMS) under the requirements of ISO 9001. The organizations generally adopted this standard in seeking competitive differentials and overcoming technical barriers (INMETRO, 2012).

In Brazil, the National Defense Strategy (END) links as a priority objective for the Brazilian Navy (MB) the denial of the use of the sea to the enemy, a complex task that, among several challenges, involves the preparation and use of its naval means. To ensure compliance with this objective, the Naval Force is empowered to design, build and operate nuclear and conventional propulsion submarines. In this search, MB developed the Submarines Development Program (PROSUB) and the Navy’s Nuclear Program (PNM) which translate into investments in personnel training, incorporation of new equipment, development of new technologies and adaptation of existing processes (Rosende & Pendone, 2016), (Brasil 2012).

Given the high level of complexity and risks of these programs, there is a need to reformulate the value chain currently established in the different MB segments and to plan the support infrastructure for Nuclear Submarine design, construction, operation, and maintenance. In this sense, SGQ ISO 9001 can internalize these requirements in the strategic processes of the military organizations of MB related to these programs.

To provide favorable conditions for transposing such challenges, MB created the Naval Agency for Nuclear Safety and Quality (AgNSNQ). A body subordinated to the Directorate General for Nuclear and Technological Development of the Navy (DGDNTM), which, in addition to contributing to the licensing of means and Nuclear naval facilities intends to assess the compliance of MB management systems, products, and services of interest.

However, the concept of certification services that AgNSNQ intends to provide to the Navy needs to be assessed from the standpoint of criteria such as feasibility, vulnerability, and acceptability (Slack et al., 2009).

Using the argument and search (model) AND (service design) in the Scielo portal we obtain 75 results, among them, only in Chimendes et al. (2008) we find a description and application of a model for the development of services, scarcity of studies on the theme (Scielo, 2018).

Chimendes et al. (2008) analyzed the models of Cowell (1988), Scheuing & Johnson (1989), Bowers (1989), Ramaswamy (1996), Tax & Stuart (1997), Bitran & Pedrosa (1998) the latter obtaining the best evaluation for the application proposed in the study: the design of an electronic ticketing service for a road transport company.

Using the evaluation of Chimendes et al (2008), this paper will use the model of Mello (2005) that compiled several authors of product and service development and proposed a process composed of four stages: Service Design Project, Process Design Service Design, Service Facilities Design and Evaluation, Improvement and Launch of the Service.

This article is structured in 6 sections and aims to evaluate the characteristics of the AgNSNQ to design the ISO 9001 QMS certification service, through the degree of attendance to the Service Design Project stage of the Service Development model proposed by Mello (2005).

After this introduction, the second section presents the general reference containing the concepts necessary for the understanding of this research. In the third section the characteristics of the case study, as well as the methods of obtaining the data according to the phases of the model, are delineated. The fourth section presented the results obtained with the characteristics of AgNSNQ, according to each phase of the service design project. In the fifth section, these results are analyzed critically against the degree of attendance to the proposed model. Finally, the sixth section presents the conclusions and proposals for future work.

2 Theoretical reference

2.1 Service design and development model

Mello (2005) critically analyzed models of the authors: Back (1983), Peters et al. (1999), Wheelwright & Clark (1992), Rosenthal (1992), Watton (1969), Vincent (1989), Pugh (1990), Ramaswamy (1996) and Bitran & Pedrosa (1998). Mello (2005) configured his model composed of the steps outlined in Figure 1 for the design and development of new services.

Figure 1
Design of the Service Design. Source: Mello (2005, p. 90 figure 4.1; p. 122, figure 4.5).

As already discussed, the design of the ISO 9001 QMS certification service is evaluated using the phases that make up the Service Design Project, including the phases of strategic analysis, idea generation and selection, service package definition and definition services specifications.

2.2. Service design project

This stage of the model searches the needs and expectations of customers for the creation of a service that meets them. Each phase identified characteristics for the project of service.

2.2.1 Strategic analysis

The strategy is aimed at listing customer needs and criteria that can generate competitive advantages for the service provider.

In its model, Mello (2005) based the strategy formulation process proposed by Gianesi & Corrêa (1994), shown in Figure 2, which begins with the analysis of two elements of the service concept being developed: segmentation (group of customers with similar characteristics) and positioning (offering differentiated services of the other competitors). After defining the market segment to be achieved and the positioning of the service, competitive criteria specific to the market should be prioritized (qualifying criteria and order winners). Then the prioritized criteria are analyzed for the definition of the strategic focus of the service and, after this phase, the service concept is defined. According to the author, the concept of the service can be defined through a succinct statement that contemplates the perceptions that the organization wishes to generate in the clients highlighting the characteristics of the service. After this definition, it is necessary to analyze the gaps (the differences between the offered service and what providers to frind the focused market follow-up). After the designers draw up the strategic aspects, they elaborate on the project schedule.

Figure 2
Formulation of the operations strategy. Source: Mello et al. (2006, figure 2).

2.2.2 Generating and selecting ideas for the service

The strategic characteristics raised in the previous item serve as input for this phase, so that, given the competitive strategy, ideas must be generated to make up the service innovation factor.

According to Sowrey (1989, apud Mello 2005), at this stage developers should use established methods to generate ideas such as: the use of advertising agencies, brainstorming, competitors' product examination, market analysis, discussion with consumer groups and interviews with customers to provide a hall of considerable insights that will be analyzed later on in terms of feasibility.

2.2.3 Service package definition

The set of related items offered to the customer contains a core or core service and other peripheral or secondary services considering the decisions of the previous phases. Mello (2005) suggests the understanding by the service developers, the parties that make up the primary service and all the processes necessary for those services to occur as planned.

Grönroos (2003 apud Mello 2005) suggests the distinction of the package into three groups of services:

1. a

   Centrals: those vital services that are the reason for the existence of the service provider;

2. b

   Facilitators: who support the central services, without them the central service cannot be consumed; and

3. c

   Support: they do not support central services, but they add value to the service.

2.2.4 Service specification definitions

According to Mello (2005), after defining the competitive strategy of the service, the differentials that the service provides and the components of the service package, the designers should focus on the users to meet the expectations of its customers. It also considers that all non-technical declarations of service customers translate design standards.

2.3 Quality management System ISO 9001

Quality Management Systems can be broken down into four concepts: systems, management, quality, and processes. Processes, according to ABNT (2015a, p. 17) “[…] a set of interrelated or interactive activities that use inputs to deliver the desired result,” these intended outcomes may be products or services. According to Senge (1999, p. 169, apud Souza, 2013, p. 20), “[…] a system is anything whose integrity and form depends on the continuous interaction between its parts.” Systems bring together processes that share the same interrelated resources to achieve a common goal.

Souza (2013) conceptualizes the term management, as being the activities that keep an organization functioning.

The quality of products and services lies in their ability to meet the expectations of customers and stakeholders of the management system (ABNT, 2015a). SGQ, therefore are processes that are interrelated to produce products and services that aim to meet the expectations of customers and stakeholders of the organization.

The fifth version of ABNT NBR ISO 9001: 2015 have seven principles: customer focus, leadership, people engagement, process approach, improvement, evidence-based decision making, and relationship management.

These principles serve to shape a simple procedural system. Identifying what customers and all stakeholders in the business want (requirements); elect a change manager (leader); involve the right professionals in the right way; map and manage processes that are part of the QMS; correct and prevent nonconformities that lead the organization to meet the requirements of its customers and decide based on the collection and analysis of data.

At each interaction, the system improves and produces better results, but the concept of risk mentality, explicit in this version, overcomes the consequences of adopting or not managing decisions (ABNT, 2015b).

Salgado & Sampaio (2013) correlated the ISO 9001 certification per 1000 inhabitants and competitiveness and innovation indices in Latin American countries and concluded that due to the proven influence on the factors that interfere in the development, SGQ ISO 9001 can be adopted as a state policy beyond the ISO 9001 standard has been tested in the most diverse economic sectors.

Maekawa et al. (2013), who analyzed a sample of 191 ISO 9001 certified Brazilian companies, demonstrated that most companies seeking certification to obtain: improved internal organization, greater productive efficiency and higher reliability in the company's brand towards its customers. The main benefits obtained were: improved quality in the processes and greater employee awareness regarding quality.

2.4 The evaluation of the ISO 9001 compliance (certification)

The quest for quality enabled the creation of conformity assessment, stimulating: fair competition, continuous quality improvement, consumer protection, facilitating foreign trade, increasing exports, protecting the domestic market and adding value to brands. Mutual recognition agreements between the results of conformity assessment programs of distinct trading partners have been established under the philosophy of “once accepted certificate” currently functioning as an instrument to facilitate trade between countries to protect against trade barriers (INMETRO, 2012).

The signatory body of the agreement recognizes the competence of conformity assessment bodies and these bodies, in turn, assess the conformity of management systems, products, processes, and persons. In Brazil, this responsibility belongs only to the General Coordination of Accreditation (CGCRE), an organ of the National Institute of Metrology, Quality, and Technology (INMETRO). CGCRE is the Brazilian body that is a signatory to the International Accreditation Forum (IAF) mutual recognition agreement (MRA), an international body that coordinates the evaluation of the signatory accreditation bodies. In general terms, the agreement ensures the recognition of certificates issued by bodies accredited by signatory bodies. (IAF, 2017).

As defined by ABNT ISO/IEC 17000: 2005, Conformity Assessment (CA) is a systematic process that has the objective of demonstrating whether organisms, systems, products, services, processes or people meet pre-established requirements with a certain degree of confidence (ABNT, 2005).

A first, second or third party can perform a CA. It is the first part when performed by the supplier of the product or service, the second part when performed by the customer and third party when performed by an independent body (ABNT, 2005).

The QMS certification is a voluntary third-party AC, described in the ABNT NBR ISO/IEC 17021-1: 2016 Conformity Assessment - Requirements for bodies that provide audit and certification of management systems, to ensure that certification bodies competent, consistent and impartial, facilitating the recognition of these organs and their certification nationally and internationally. In the case of bodies that certify ISO 9001 (OCS) ISO/IEC 17021-3: 2017 Conformity assessment - Requirements for bodies providing audit and certification of management systems - Part 3: Competence requirements for auditing and certification of quality management systems, still without the Portuguese version, specify the competency criteria required for QMS auditors (ISO, 2017b).

3 Methods

To evaluate the design of the ISO 9001 QMS certification service to be offered by AgNSNQ, a single case study was carried out at the Agency.

3.1 Purpose of the case study

This case study, characterized as a single case, aimed to evaluate the characteristics of AgNSNQ to design the ISO 9001 QMS certification service through the degree of service to the Service Design Project stage (item 2.2). Scores from 1 to 5 were assigned according to AgNSNQ's ability to meet the criteria of each phase.

3.2 Data collection techniques

To obtain the data were performed:

• Observations as a participant: when the researcher assumes a role in the life of the organization.

• Bibliographic research: in books, magazines, newspapers, theses, dissertations, and annals of scientific and material events made available in public sources; and

• Documentary research: carried out through the analysis of documents prepared for diverse purposes such as settlements, authorizations, and communications (Gil, 2017).

The databases consulted that did not return results according to the selection criteria were not cited in the results.

3.2.1 Methods for strategic analysis of the service

The criteria were subdivided into lines of approach because of the complexity of the data collection for this phase. The design phase of the project schedule is not addressed in this study.

1. a

   Segmentation and positioning

The purpose of the creation of the AgNSNQ indicates that the certification service should only serve the Military Organizations (MO) of MB and especially those wh willo belong to the infrastructure of support to the SN-BR using this workforce of the AgNSNQ.

These characteristics, therefore, indicate the segmentation and positioning of the service. A documentary survey was carried out to obtain ISO 9001 QMS certificates present in MO of MB and AgNSNQ's ability to provide the service to these MOs (according to the scope defined in the MO certificate and location).

The research was carried out using the Certification Portal (Certifiq), a database that congregates accredited bodies and the certificates issued for SGQ 9001 in the country, administered by CGCRE. The portal lists the certificates issued by the accredited bodies in the country, the data of the certified organizations and the scope of the QMS.

A table presented the research data in the Certifiq portal it's containing: the MO of the MB that has ISO 9001 certified QMS, the certifying body, the scope of the QMS, the degree of attendance to the segmentation criterion and a note for AgNSNQ's ability to attend. Considering the organization's potential to belong to the SN-BR support infrastructure, the scope and distance of the MO from RJ to provide the service). The scores attributed to the criteria of segmentation and positioning varied from 1 to 5, where: 1 is the inadequacy of the QMS to meet the criterion, and 5 is the total suitability of the QMS to meet the criterion. Data collection took place in June 2018.

1. b

   Definition of competitive criteria and focus analysis

To identify competitive criteria inherent to the market, we searched for bibliographical sources that addressed criteria for choosing ISO 9001 certification bodies. The bases used were the Scielo portal and the thesis and dissertations catalog of Capes. The searches were conducted in February 2018 and the search arguments used were:

• On the SciELO portal: (certification) AND (ISO 9001) AND (quality); and

• In the Capes thesis and dissertations catalog: “Quality Management System” AND “Certification” AND “Audit”;

The results were compiled in a table containing the base, the number of results found, the title of the results found, the author (s), a summary of the publication, the competitive criteria identified; notes attributed to the ability of AgNSNQ to compete in the criterion and the justifications for awarding the note. The criteria in which the service obtained greater indicated the competitive focus. The search was conducted in November 2018.

1. c

   Definition of Service Concept

The service concept was synthesized in a statement containing the service description highlighting: the follow-up, the positioning and the competitive criterion in which the service focused.

1. d

   Analysis of the gaps in the services offered by the market

To identify possible discrepancies between the planning of the audit service and those provided, we sought authors who presented critical analyzes of the QMS audit process.

The research was carried out in the Catalog of theses and dissertations of CAPES using the arguments: “critical analysis” AND “audit” AND “quality management system.”

The results were presented in a table containing the results found, the title of the selected result, the author (s), a summary, identified gaps, the particular note and a justification of the assigned grade. Assigned scores ranged from 1 to 5 were: 1 when AgNSNQ auditors had no potential to fill the gap and 5 when AgNSNQ auditors would have full potential to fill the gap. This research was conducted in November 2018.

3.2.2 Methods for generating ideas for the service

The results obtained with the strategic analysis of the service were examined to identify the characteristics of the AgNSNQ that could originate innovations for the service. A specific table presented ideas generating characteristic and notes varying from 1 to 5 for the criteria: desirability (attending to a desire or need of the user), feasibility (there is technology available to implement the idea), feasibility budget for the realization of the idea) and a column containing justification for the notes (Lopes, 2017).

3.2.3 Methods for assembling the service pack

According to Slack et al. (2009), the definition of the components of the service or product package stems from the collection of information about the order in which the components are assembled.

Because the service is standardized, the services (central, facilitators and support) were detailed through the requirements for ABNT NBR ISO 17021-1: 2016 processes as discussed in item 2.4.

The processes related to the standard were classified in services: central, facilitators and support. Each process indicated in the standard was compared to the implementation capacity in AgNSNQ. Notes 1 to 5 were assigned where: 1 is the inability to implement the process and 5 when the Organization already has a similar process implemented.

A table was drawn up containing: the process, the assigned grade and a justification of the grade for evaluation of attendance to the criterion.

3.2.4 Methods for service specifications

We conducted searches on the Google Scholar base using the search query: “customer satisfaction” AND “ISO 9001 audit” to collect recurring requirements from customers for ISO 9001 certification services in the base publications. The requirements were analyzed in contrast to the characteristics of the AgNSNQ to identify its capacity to instill in its service the specifications demanded by the market. The results of the evaluations were synthesized in a table containing: the results found, a brief synopsis of the result, the specifications indicated, the characteristics present in the AgNSNQ, and a note indicating the degree of compliance with the specification. Grades ranged from 1 to 5 where 1 is the inability of the feature to insert the specification into the service and 5 the capability of the feature to fully insert the specification into the service. The research was conducted in November 2018.

3.3 Analysis and interpretation of results

After the presentation of the results, a summary table containing the averages obtained by the characteristics of AgNSNQ services was elaborated at each phase of the service design project. The criterion of the strategic analysis “Definition of the concept of service” was treated if it answers or does not answer (1 or 5), the others had their means analyzed. The means were classified into intervals varying from 1 to 5 which meant: from 1 to 2 - the characteristics found did not meet the criterion; from 2.1 to 3 - the characteristics found need a lot of effort to meet the criterion; from 3.1 to 4 - the characteristics found need little effort to meet the criterion and from 4.1 to 5 - the characteristics meet the criterion. The design of the service was conditioned to the service in all the criteria.

4 Results

This item has the results found according to the requirements of the service design project model.

4.1 Organization characterization

According to Santos & Muniz (2011), since 2002, Ministry of Defense (MD) Working Groups (WG) study the centralization of metrology, standardization, and conformity assessment activities, following MD guidelines. Such demand arose because of the strategic importance of these activities to the Industrial Defense Base (IDB): public and private organizations that participate in the research, development, production, distribution, and maintenance of strategic defense products. In the 1980s, the Brazilian Air Force (FAB) incorporated activities to evaluate compliance with the implementation of the Aerospace Metrology System (SISMETRA), a reference for the WG, due to the benefits obtained in the tests, calibrations, the technical quality of the technological processes and the certification of persons. Members concluded that a centralized MD body should be created dedicated to coordinating CA activities within the MD. The WG further suggested that this body could accredit the bodies within the MD, i.e., recognize the competence of these bodies to assess the compliance of systems, products, and people.

The MO of the MD focused solely on assessing the conformity of strategic products and services to the IDB has not been created, but since 2010, the Defense Products Secretariat (SEPROD) of the MD has been responsible for monitoring studies and strategic projects in the IDB, which concerns the technological feasibility, development, procurement, and applicability of a defense product. Currently, SEPROD promotes the search for such competencies in the three forces to maintain and develop the dissuasive power of the national Armed Forces (Santiago, 2014).

Following these MD initiatives, the High Naval Administration decided to form a WG to study the activities carried out by the Institute of Development and Industrial Coordination (IFI) of the FAB in order to present possible advantages and disadvantages of a similar body in the MB and that served as interlocutors with the SEPROD of the MD.

The IFI is an MO that assesses the compliance of management systems and plays a strategic role in consolidating the strategic objectives of the Brazilian Air Force (Cunha & Alves, 2008). The General Secretariat of the Navy (SGM) issued an order establishing the WG-IFI, formed by representatives of military organizations related to the activities of science, technology and materials of MB, which after meetings and internal research, issued report containing the group's impressions on what functions could be performed by an MB's MO for conformity assessment. In 2012, the Nucleus of Implementation of Technological and Industrial Development Office of MB (NuEDTI) was created, to be structured progressively according to budget availability.

Initially installed in the premises of the Research Institute of the Navy (IPqM) under the administration of the then Secretariat of Science, Technology and Innovation of the Navy (SecCTM), NuEDTI would be responsible for carrying out conformity assessment activities indicated by the WG within the scope of MB; initiative reinforced by the public NED in 2013 (Brasil, 2013)

Continuing the activation activities of EDTI in 2015, SecCTM entered into an agreement of mutual intentions with INMETRO aiming at cooperation regarding technological solutions, methods, and processes related to metrology, quality, and conformity assessment. At the same time, MB passed and sent officers to study in the areas of Metrology and Quality at INMETRO and at the Pontifical Catholic University of Rio de Janeiro (PUC-Rio), as well as taking courses offered by the IFI to train its staff.

To give dynamism and optimization of resources, in mid-2016, the High Naval Administration decided to restructure the areas of science and technology in the Navy by changing the name of the SecCTM to the Directorate General of Nuclear and Technological Development (DGDNTM), which started to manage PROSUB and the PNM.

Just like appointed by Guimarães (1999) and due to the advancement of these programs, the need for an element to contribute to the licensing activities of nuclear naval installations, due to the novelty of the nuclear submarine project and technical peculiarities related to nuclear energy applied to naval propulsion.

Due to the similarities of these activities with those of conformity assessment, it was concluded that NuEDTI's activities should cover the naval nuclear area, and its name changed to Nucleus for the Naval Agency for Nuclear Safety and Quality (NuAgNSNQ). The AgNSNQ, which was activated on February 5, 2018, is located in the Cobras Island Naval Complex (CNIC) - Rio de Janeiro / RJ, and currently has 5 departments, namely: Analysis of Projects and Norms; Analysis of Nuclear Safety Operations; Metrology, Quality, and Certification; Planning and Organizational Development; and, Administration, as shown in Figure 3:

Figure 3
Organogram’s AgNSNQ. Source: Elaborate by authors.

Conformity assessment activities carried out by the Department of Metrology, Quality and Certification (AgNSNQ-30), which has two divisions: Metrology and Quality and Certification, the latter designed to evaluate the conformity of management systems and MB products of interest. Currently, the Certification Division has quality master engineers from the Pontifical Catholic University of Rio de Janeiro (PUC-RJ) and INMETRO.

The officers develop research projects aligned with the future activities of the divisions. It is planned that these officers act as ISO 9001 QMS auditors. The Organizational Planning and Development Department prepares internal selection processes to participate in the selection process, and candidates must be from the Navy’s Corps of Engineers (CEM) and be at least in the position of Captain-Lieutenant (at least 6 years of experience). The selected military personnel attend INMETRO or PUC. As a result of agreements signed with the IFI, the officers of the Certification Division (AgNSNQ-32) participate, as observers, in audits carried out in Rio de Janeiro. As a prerequisite for participation in IFI audits, the officers are required to interpret ISO 9001: 2015 and audit 19011: 2012 - Guidelines for management system audit. The IFI ministers both courses and each of the officers have already participated in at least one audit as an observer.

4.2 New service characterization

AgNSNQ plans to develop the ISO 9001 QMS certification service (item 2.3) following the requirements of ISO 17021-1: 2016 (item 2.4) by the Certification Division officers. The certification contributes to the preparation of the infrastructure to support the operations of the first nuclear submarine of the Brazilian Navy (Brasil, 2018).

4.3 Results of the strategic analysis of the service

4.3.1 Segmentation and positioning

Table 1 brings together the MB military organizations that have SGQ ISO 9001:

Table 1
Military Organizations of MB with certified QMS.

Military Organization	UF	Certification Body	Scope	S	P
Arsenal de Marinha do Rio de Janeiro	RJ	BVQI DO BRASIL SOCIEDADE CERTIFICADORA LTDA	Manufacture of tubes, pipes, hollow profiles, and associated steel fittings	5	5
	RJ		Other consulting, scientific, technical and similar activities, n.e.	5	5
	RJ		Activities auxiliary to water transport	5	2
Caixa de Construções de Casas para o Pessoal da Marinha	RJ	ICQ BRASIL - INSTITUTO DE CERTIFICACAO QUALIDADE BRASIL	Other financial service activities, except insurance and pension funds, n.e.	2	1
Capitania dos Portos de Alagoas	AL	FCAV - FUNDAÇÃO CARLOS ALBERTO VANZOLINI	Technical and vocational secondary education	2	2
Capitania dos Portos de Sergipe	SE	ABNT - ASSOCIAÇÃO BRASILEIRA DE NORMAS TÉCNICAS	Other educational activities, n.e.	2	2
	SE		Other educational activities, n.e.	2	2
Capitania dos Portos do Ceará	CE	BRTUV AVALIAÇÕES DA QUALIDADE S.A.	Sea driving and driving schools	2	2
	CE		Sports and recreational education	1	2
Capitania Fluvial do Araguaia	TO	FCAV - FUNDAÇÃO CARLOS ALBERTO VANZOLINI	Sea driving and driving schools	1	1
Centro de Análise de Sistemas Navais	RJ	LLOYDS REGISTER DO BRASIL LTDA	Other consulting, scientific, technical and similar activities, n.e.	5	5
	RJ		Other telecommunications activities	5	5
Centro de Instrução Almirante Graça Aranha	RJ	ABNT - ASSOCIAÇÃO BRASILEIRA DE NORMAS TÉCNICAS	Technical and vocational secondary education	4	3
Capitania dos Portos do Espírito Santo	ES	ABNT - ASSOCIAÇÃO BRASILEIRA DE NORMAS TÉCNICAS	Other educational activities, n.e.	2	2
	ES		Technical and vocational secondary education	2	2
	ES		Other educational activities, n.e.	2	2
	ES		Other educational activities, n.e.	2	2
Capitania dos Portos do Rio de Janeiro	RJ	ABNT - ASSOCIAÇÃO BRASILEIRA DE NORMAS TÉCNICAS	Technical and vocational secondary education	3	2
	RJ		Sports and recreational education	3	2
	RJ		Teaching of cultural activities	3	1
	RJ		Other educational activities, n.e.	3	2
Delegacia da Capitania dos Portos em Macaé	RJ	ABNT - ASSOCIAÇÃO BRASILEIRA DE NORMAS TÉCNICAS	Technical and vocational secondary education	2	2
Centro Tecnológico da Marinha em São Paulo	SP	BVQI DO BRASIL SOCIEDADE CERTIFICADORA LTDA	General mechanical activities	4	5
Average				2.91	2.61

• Source: INMETRO (2018). Legend: P = consistent with the proposed segmentation; S = ability to meet the scope.

4.3.2 Definition of competitive criteria and focus analysis

Table 2 shows the results that met the selection criteria established for publications that contain reports on the established criteria:

Table 2
Competitiveness criterion and focus analysis.

Base	Results found	Selected Result	Author	Synthesis	Criterion	Score	Justification
Scielo	15	A multi-criteria approach to a service consultancy and certification of Quality Management Systems	Silva & Melo (2017)	Paper of Gestão & Produção Journal in which the authors elaborated a multicriteria methodology for the qualification and selection of Consultants for the implementation of ISO 9001 QMS and accredited ISO 9001 QMS Certification Bodies.	Cost	5	Because the organization has its operating costs borne by the Union, because not depend on income from certification activities to sustain itself.
					Payment methods	5	Because the organization has its operating costs borne by the Union, because not depend on income from certification activities to sustain itself.
					Transparency	5	For the planned market segment (MO of MB) the organization can act with total transparency without fear of exposing intellectual property.
					Ability to Answer Complaints	1	Currently, the body has no experience to meet the criteria.
					Certification Proposal	1	Currently, the body has no experience to meet the criteria.
Capes thesis and dissertations catalog	9	The impact of certification on the quality of the Brazilian aerospace sector from the perspective of certified organizations	Rambo (2011)	Master's thesis in which the author evaluated 18 factors that interfere in the audits of QMS in the aerospace sector. Among the factors, 5 are related to the performance of the auditors.	Technical competence and personal attributes of the auditor;	3	Although the body of auditors have a higher level and a postgraduate degree in Metrology and Quality, they have little experience in audits of QMS ISO 9001
					Prior analysis of the organization's QMS;	3	MB has few organizations and few types that deliver similar products and services, it is believed, therefore, that auditors did not have difficulty to understanding the QMS of these organizations; however, it does not currently have the experience to review QMS in the light of ISO 9001 critically
					The presentation of findings and conclusions of the audit;	2	Auditors' knowledge of this criterion is essentially theoretical
					The critical analysis and dissemination of audit results;	2	Auditors' knowledge of this criterion is primarily theoretical.
					Take note of the results during and at the end of the audit;	2	Auditors' knowledge of this criterion is primarily theoretical.
Average						2.9

• Source: adapted from Silva & Melo (2017); Rambo (2011) e Brasil (2018)

4.3.3 Concept of service

Given the positive results of MO of MB having ISO 9001 QMS (justifying segmentation), the premise of offering certification services using AgNSNQ's auditors (positioning), the favorable competitiveness criteria (cost, payment methods, and transparency) the service concept can be formulated according to the statement below:

“AgNSNQ will provide ISO 9001 QMS certification services for MB's MO. Your auditors will be able to understand the peculiarities of the low-cost segment and in a transparent way.”

4.3.4 Gaps

Table 3 presents the classification of the main gaps and the capacity of AgNSNQ to fill them.

Table 3
Gaps in the audit process.

Results found	Selected Result	Author	Synthesis	Identified gaps (explanation)	Score	Justification
4	Critical analysis of the audit process of the quality management system in the aerospace sector	Souza (2010)	The author interviewed auditors to identify which requirements are ranked as the most important and which have the highest compliance rate in practice in the QMS audit process. Through a Pareto chart, the author identified the 5 most significant gaps between the most critical requirements and their levels of application in practice.	Selection of the audit team: indicates that greater attention should be given to the planning of the audit, as to the skills required for the auditors selected to perform the audit.	3	Considering the academic qualification criteria, the AgNSNQ (postgraduate) auditors have the potential to fill this gap; however, the low and experience in QMS audits prevents filling this gap.
				Top management commitment: it shows that, in the perception of the auditors, the commitment of the top management of the organizations is precarious but fundamental for the audit to reach the desired effect.	1	The AgNSNQ has no potential to promote a more significant commitment of the Senior Management of the audited organizations because involved in a hierarchical environment
				Improvement of the audit process: denotes the need to establish a more effective system for the continuous improvement of the audit process from both the auditee and INMETRO evaluators.	1	Because it has little experience, this gap has a low potential to be filled by AgNSNQ auditors
				Previous analysis of the organization's QMS: shows that there is a potential need to improve the prior analysis of the organization's QMS by understanding its products and processes adequately.	3	MB has few organizations and few types that deliver similar products and services, it is believed, therefore, that auditors did not have difficulty understanding the QMS of these organizations; however, it does not currently have the experience to critically analyze QMS in the light of ISO 9001
				Communication during the audit: the ability to communicate effectively through the various hierarchical levels of the organization audited during audits.	5	Because it belongs to the same institution (MB), it is expected that the AgNSNQ auditors don't have difficulties communicating with the audited organizations
Average					2.6

• Source: Adapted from Souza (2010).

4.4 Results of the ideas for the service

Table 4 contains the characteristics of the organization, the ideas generated and the individual evaluation according to the specific criteria.

Table 4
Ideas for the service.

Feature	Innovation	D	P	V	Average	Justification
Independence of revenue from services rendered	Free service	5	5	3	4.34	Because the organization is not for profit, the service can be offered free of charge, is the responsibility of the audited the costs with the transfer and hosting of the auditors vis-a-vis SGQ outside RJ (AgNSNQ headquarters)
	Detailed audits	1	5	1	2.34	Given the lack of pressures to minimize audit times, these would prioritize compliance; however, the auditees would prefer less time-consuming audits to minimize the impact on the organization's operation and the values ​​with hosting audit staff
Auditors of the institution itself	Procedures Using MB Terms and Codes	5	4	5	4.67	The auditees would have a natural understanding of certification procedures due to the use of terms and codes used by the institution; however, this customization could create confusion for INMETRO evaluators
	Audits following the MB secrecy requirements	5	5	5	5	Since the auditors belong to the same institution, the probability of leakage of sensitive information from organizations involved with the SN-BR project is small
The body belonging to the same institution as the audited	Full access to the body's procedures	3	2	4	3	Despite the facility for auditees to access procedures for certification activities, routine procedural revisions could lead to a gap between current procedures and those available on the sharing platform
Average General					3.87

• Source: Prepared by the authors. Legend: D = desirability; P = Practicality; V = viability.

4.5 Results for service packs

According to ABNT NBR ISO 17021-1: 2016 (item 2.4), the processes indicated in the standard were classified as central, facilitating, and supporting services.

4.5.1 Central services

The processes that characterize the provision of the certification service were considered central services:

1. a

   The critical review of documentation: involves evaluation if the scope of the client's QMS is assisted by the scope of accreditation of the agency;

2. b

   Audit program planning: A program involving the complete certification cycle (three years) is prepared and the audit plan for phase 01 (preliminary visit). This information is sent to the client who must accept or reject the proposal;

3. c

   Phase 01 audit planning: selection and appointment of the audit team and elaboration of the knowledge audit plan;

4. d

   Conduct of Phase 01: knowledge audit of the QMS and solution of the areas of concern of the knowledge audit;

5. e

   Phase 02 audit planning: selection and designation of the audit team and preparation of the verification audit plan;

6. f

   Conduct of Phase 02: conduct the audit to verify the effectiveness of the client's QMS;

7. g

   Certification Decision: granting of initial certification and issuance of certification documents;

8. h

   Supervision audits: exchange of information between client and certification body (example: change of scope), determination of whether changes are not necessary for the audit program, confirmation of the audit program and conduction of the audit of the portion of the processes (at the end of the two oversight audits the entire QMS should have been assessed);

9. i

   Recertification: it complies with the provisions of a) to g).

Figure 4 illustrates a flowchart involving central services related to ISO 9001:

Figure 4
A flowchart of the certification process. Source: adapted from ABNT (2016), Figure E.1.

4.5.2 Facilitating services

Facilitating services were considered to be processes that make central services viable:

1. a

   Process for the management of impartiality: according to item 5.2.3, the organization must have a process to identify, analyze, evaluate, treat, monitor and document the risks related to conflicts of interest related to the audit activities;

2. b

   Process for the competence of personnel: according to item 7.1, the organization must have processes that guarantee: competence criteria, initial evaluation and continuous monitoring of competence (selection, training, and appointment of auditors), selection and familiarization of specialists and registration of all staff;

3. c

   The process to guarantee confidentiality: according to item 8.4.7, the organization must have processes that guarantee the security of the confidential information;

4. d

   Process for Suspension, Cancellation or reduction of the scope of certification: according to item 9.6.5, the organization must have documented policy and procedures for suspension, cancellation or reduction of certification scope as well as subsequent actions;

5. e

   Appeals: According to item 9.7, agencies must establish procedures to receive, evaluate and make decisions on appeals of their clients;

6. f

   Complaints: According to item 9.8, the organizations must have a specific process to receive, evaluate and make decisions and make decisions on complaints, from any interested party of their services; and

7. g

   Clients Registration: agencies must keep records of their audits and other certification activities for all their clients, including organizations that have requested the service.

4.5.3 Support services

Support services were considered to be processes related to the internal QMS of the organization that may choose to have an ISO 9001 QMS or implement and maintain the processes below:

1. a

   Document and Records Control Process: the certification body must establish procedures to control internal and external documents and records related to the certification of Management Systems;

2. b

   Critical Management Review Process: senior management should establish procedures to analyze the effectiveness of their QMS at planned intervals critically;

3. c

   Internal Audit process: the body shall establish internal audits to verify that it meets the requirements of the standard; and

4. d

   Process for corrective actions: the body must establish requirements for identification, management and elimination of causes of nonconformities;

4.5.4 Process implementation capacity

Table 5 compiled the grades assigned to the process implementation capacity.

Table 5
The ability to perform the services.

Process	Score	Justification
Critical Review of QMS Documentation	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Audit program planning and certification contract	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Audit Planning for Phase 01	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Conducting Phase 01	3	Auditors have theoretical and practical knowledge, but the experience is lacking.
Phase 02 Audit Planning	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Conducting Phase 02	3	Auditors have theoretical and practical knowledge, but the experience is lacking.
Certification Decision	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Certificate Issuance	3	Although they only have theoretical knowledge about this process, it is of low complexity
Supervision Audits	3	Auditors have theoretical and practical knowledge, but the experience is lacking.
Recertification	3	Auditors have theoretical and practical knowledge, but the experience is lacking.
Management of Impartiality	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Competence of staff	5	AgNSNQ has an established process for the selection, qualification, and evaluation of its personnel
Confidentiality guarantee	4	AgNSNQ officers are subject to strict confidentiality procedures that can be used for the certification service
Suspension, cancellation or reduction of the scope of certification	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Appeals	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Claims	2	Os auditores possuem apenas conhecimentos teóricos a respeito dos critérios avaliados neste processo.
Customer registration	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Control of Documents and Records	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Critical review by senior management	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Internal audits	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Corrective actions	2	The auditors only have theoretical knowledge regarding the criteria evaluated in this process.
Average	2.43

• Source: Elaborate by authors.

4.6 Results for service specifications

The results obtained in the database are shown in Table 6, according to the established criteria.

Table 6
Service specifications.

Results	Author	Synthesis	Specifications	Score	Justification
The critical review of the quality audit process	Almeida (2012)	The author carried out a case study at a company where he collected data on the contribution of the audit process to the verification of the effectiveness of the organization's QMS	Experience	2	AgNSNQ auditors have little experience with QMS audits
			Good oral and written communication	4	The military training centers focus on oral and written communication skills as well as these skills are practiced during the career
			Technical knowledge of the standard and the audited process	2	The technical knowledge of the AgNSNQ auditors' standard is primarily theoretical; however, they have good prior knowledge of the auditees
			Leadership and ethical posture	3	The military career imposes as performance requirements criteria such as leadership and ethics, however, such characteristics were not applied during QMS audit processes
Audits of management systems: skills for adding value	Soratto da Silva (2011)	The author identified a framework of audited and audited knowledge, skills, and attitudes that add value to audit processes	Knowledge in the scope of the audit and the processes to be audited	4	AgNSNQ auditors have good experience of the processes of MB organizations that are audited, but this expertise has never been applied in QMS audits
			The significance of findings, opportunities for improvement and nonconformities	2	As they have never participated as lead auditors in QMS audits, auditors currently only have theoretical knowledge regarding the criterion
			Useful report and appropriate analysis of the effectiveness of the corrective actions implemented by the auditee	2	As they have never participated as lead auditors in QMS audits, auditors currently only have theoretical knowledge regarding the criterion
Average				2.71

• Source: Adapted from Almeida (2012); Soratto da Silva (2011).

5 Analysis of the results and application of the model

Table 7 shows that only the criterion of “Definition of the service concept” met the criterion indicating that the service declaration can be linked as the definition of differentiated service.

Table 7
Evaluation of meeting the criteria.

Design Phase of the Service Design Project	Average	Ranking
Strategic Analysis (item 4.3)
Segmentation (item 4.3.1)	2.91	With a great effort of the organization, this criterion can be taken care of
Positioning (item 4.3.2)	2.61	With a great effort of the organization, this criterion can be taken care of
Definition of Competitiveness Criteria and Focus Analysis (item 4.3.3)	2.9	With a great effort of the organization, this criterion can be taken care of
Definition of the Service Concept (item 4.3.4)	5	The characteristics found to meet the established criteria
Analysis of the gaps in the services offered by the market (item 4.3.5)	2.6	With a great effort of the organization, this criterion can be taken care of
Ideas for the service (item 4.4)	3.87	With little organization effort, this criterion can be met
Service Package (item 4.5)	2.43	With a great effort of the organization, this criterion can be taken care of
Specifications for the Service (item 4.6)	2.71	With a great effort of the organization, this criterion can be taken care of

• Source: Prepared by the authors.

Only the criterion “ideas for service” was classified with little effort to meet the criterion showing that AgNSNQ has characteristics that can promote innovations for the service. Efforts to provide timely audits by competitors may raise the average obtained in the criterion, for the other criteria were identified characteristics that with much effort of the developers can meet the criterion.

Regarding the segmentation and positioning criteria, it was possible to collect data from 13 ISO 9001 QMS assets in MO of the MB according to the portal Certifiq indicating that there are clients that justify the segmentation defined by the organization's strategy. However, it is observed the predominance of QMS aimed at the training of professionals and amateurs for waterway traffic. Few Military Service Organizations (MSO) (Navy’s Arsenal in Rio de Janeiro, Center for Navy’s Systems Analysis and Navy’s Technology Center in São Paulo) that provide maintenance and scientific services can be classified as part of an eventual infrastructure of support to the design, construction and operation of SN-BR. The disclosure of ISO 9001 certification services could boost the demand for other MOPS for these services.

Regarding the definition of competitive criteria and focus analysis, it can be seen that the low experience of the auditors and the absence of processes for the composition of certification proposals and responses to the clients place the average in levels that indicate the fulfillment of the criterion only with many efforts. This criterion indicates that the service should focus on parameters such as cost and transparency and that auditors should acquire more experience and seek training in these processes.

The low grades attributed to the specifications that should be internalized at the service and the analysis of the main gaps observed in audits are strictly linked to the experience level of the auditors; however, the highest scores in these two criteria were attributed to the profile of the auditors inherent in a military career.

There is a close link between the low scores obtained in the service package analysis and the low experience of the auditors with the ISO 9001 QMS certification processes. It is noteworthy the note obtained for the training process that has been recruiting military personnel for the composition of the Department of Planning and Organizational Development.

Finally, the results indicate that no criterion obtained a score below 2.1 (does not meet the criterion).

6 Conclusions

In the analysis of AgNSNQ's creation history, it is realized that significant challenges must be overcome for the organization to achieve the purposes of its creation. However, given its recent creation and the characteristics it already has, it is understood that in the short term it can promote the reliability of products and services of the infrastructure supporting the SN-BR through the certification and ISO 9001 QMS.

It is concluded that, through the evaluation of the degree of attendance of the current characteristics of AgNSNQ to the criteria of the Service Design Project stage. The ISO 9001 QMS certification service does not meet the requirements to be designed since in most of the criteria it was obtained an indication that the organization must make great efforts to meet them. The noun practical experience of the auditors can be pointed out as the main factor for not meeting the criteria. In a particular analysis to the positioning criterion, the low number of clients targeted for the service (MO of the SN-BR support infrastructure) is observed, however, the MOs that already have active QMS can benefit from the certification service offered by the AgNSNQ.

Despite these shortcomings, there is an excellent innovation potential for an eventual ISO 9001 certification service offered by AgNSNQ.

Despite the indication for not designing the service, the model could indicate lines of action that can be adopted by the organization to obtain favorable evaluations for the design of the service.

Future work may address ISO 9001 QMS auditors' qualification tracks and how be developed by AgNSNQ and other stakeholders can develop them.

• Financial support: None.
• How to cite: Sousa, A. A. G., Lima, M. P. S., & Fermam, R. K. S. (2020). The design of the certification service of quality management systems ISO 9001: a case study in a military organization of the Brazilian Navy. Gestão & Produção, 27(4), e4715. https://doi.org/10.1590/0104-530X4715-20

References

• Almeida, P. A. O. (2012). Análise crítica ao processo de auditoria da qualidade (Dissertação de mestrado). Instituto Superior de Contabilidade e Administração do Porto, Porto.
• Associação Brasileira de Normas Técnicas – ABNT. (2005). ABNT NBR ISO/IEC 17000:2005: avaliação da conformidade: vocabulários e princípios gerais Rio de Janeiro: ABNT.
• Associação Brasileira de Normas Técnicas – ABNT. (2015a). ABNT NBR ISO 9000:2015: sistemas de gestão da qualidade: fundamentos e vocabulário Rio de Janeiro: ABNT.
• Associação Brasileira de Normas Técnicas – ABNT. (2015b). ABNT NBR ISO 9001:2015: sistemas de gestão da qualidade: requisitos Rio de Janeiro: ABNT.
• Associação Brasileira de Normas Técnicas – ABNT. (2016). ABNT ISO/IEC NBR 17021-1:2016: avaliação da conformidade: requisitos para organismos que fornecem auditoria e certificação de sistemas de gestão Rio de Janeiro: ABNT.
• Back, N. (1983). Metodologia de projeto de produtos industriais. Rio de Janeiro: Guanabara Dois.
• Brasil. (2012). Estratégia Nacional de Defesa (END) Brasília: Ministério da Defesa. Retrieved in 2018, March 11, from http://www.defesa.gov.br/arquivos/estado_e_defesa/END-PND_Optimized.pdf
  » http://www.defesa.gov.br/arquivos/estado_e_defesa/END-PND_Optimized.pdf
• Brasil. (2013, 25 de setembro). Decreto-Lei nº 373, de 25 de setembro de 2013. Aprova a Política Nacional de Defesa, a Estratégia Nacional de Defesa e o Livro Branco de Defesa Nacional (seção 1, pp. 1). Brasília, DF: Diário Oficial da República Federativa do Brasil.
• Brasil. Marinha do Brasil. (2018, 26 de março). Portaria nº 46 da DGDNTM de 26 de março de 2018. Regulamento da Agência Naval de Segurança Nuclear e Qualidade Brasília, DF: Diário Oficial da República Federativa do Brasil.
• Bitran, G., & Pedrosa, L. (1998). A structured product development perspective for service operations. European Management Journal, 16(2), 169-189.
• Bowers, M. R. (1989). Developing New Services: improving the process maker in the better. Journal of Services Marketing, 3(1), 15-20.
• Chimendes, V. C. G., Mello, C. H. P., & Paiva, A. P. D. (2008). Análise de modelo para projeto e desenvolvimento de serviços: uma pesquisa-ação em uma empresa de transporte rodoviário de passageiros. Gestão & Produção, 15(3), 491-505. http://dx.doi.org/10.1590/S0104-530X2008000300006
  » http://dx.doi.org/10.1590/S0104-530X2008000300006
• Cowell, D. W. (1988). New service development. Journal of Marketing Management, 3(3), 296-312.
• Cunha, L. O., & Alves, J. M. (2008). Um organismo acreditado de certificação de sistemas de gestão da qualidade contribuindo para a garantir a segurança de voo. In Anais do XXVIII Encontro Nacional de Engenharia de Produção São Paulo: Enegep.
• Fernandes, W. A. (2011). O movimento da qualidade no Brasil Brasil: Essential Idea Publishing.
• Gianesi, I. G. N., & Corrêa, H. L. (1994). Administração estratégica de serviços: operações para a satisfação do cliente São Paulo: Atlas.
• Gil, A. C. (2017). Como elaborar projetos de pesquisa (6. ed.) São Paulo. Atlas.
• Guimarães, L. S. (1999). Síntese de doutrina de segurança para projeto e operação de submarinos nucleares (Tese de doutorado). Escola Politécnica, Universidade de São Paulo. São Paulo.
• Instituto Nacional de Metrologia, Qualidade e Tecnologia – INMETRO. (2012). Avaliação da conformidade Retrieved in 2018, March 11, from http://www.inmetro.gov.br/inovacao/publicacoes/acpq.pdf
  » http://www.inmetro.gov.br/inovacao/publicacoes/acpq.pdf
• Instituto Nacional de Metrologia, Qualidade e Tecnologia – INMETRO. (2018). Portal certifique: consulta empresas Retrieved in 2018, January 5, from http://certifiq.inmetro.gov.br/Consulta/ConsultaEmpresas
  » http://certifiq.inmetro.gov.br/Consulta/ConsultaEmpresas
• International Accreditation Forum – IAF. (2017). IAF members & signatories Retrieved in 2017, December 27, from http://www.iaf.nu/articles/IAF_MEM_Brazil/59
  » http://www.iaf.nu/articles/IAF_MEM_Brazil/59
• International Organization for Standardization – ISO. (2017a). The ISO survey of management system standard certifications 2016 Retrieved in 2018, March 11, from https://isotc.iso.org/livelink/livelink/fetch/-8853493/8853511/8853520/18808772/00._Executive_summary_2016_Survey.pdf?nodeid=19208898&vernum=-2
  » https://isotc.iso.org/livelink/livelink/fetch/-8853493/8853511/8853520/18808772/00._Executive_summary_2016_Survey.pdf?nodeid=19208898&vernum=-2
• International Organization for Standardization – ISO. (2017b). IEC 17021-3:2017: conformity assessment: requirements for bodies providing audit and certification of management systems: part 3: competence requirements for auditing and certification of quality management systems Genebra: ISO.
• Lopes, S. (2017). Fizemos o brainstorming. E agora? Como priorizar as ideias? Retrieved in 2018, November 21, from https://www.linkedin.com/pulse/fizemos-o-brainstorming-e-agora-como-priorizar-ideias-sonia-lopes-
  » https://www.linkedin.com/pulse/fizemos-o-brainstorming-e-agora-como-priorizar-ideias-sonia-lopes-
• Maekawa, R. M., Carvalho, M. M., & Oliveira, O. J. (2013). Um estudo sobre a certificação ISO 9001 no Brasil: mapeamento de motivações, benefícios e dificuldades. Gestão & Produção, 20(4), 763-779. http://dx.doi.org/10.1590/S0104-530X2013005000003
  » http://dx.doi.org/10.1590/S0104-530X2013005000003
• Mello, C. H. P. (2005). Modelo para projeto e desenvolvimento de serviços (Tese de doutorado). Universidade de São Paulo, São Paulo.
• Mello, C. H. P., Costa Neto, P. L. D. O., & Turrioni, J. B. (2006). Modelo para o projeto e desenvolvimento de serviços: uma proposta brasileira. Revista GEPROS, (61), 61.
• Peters, A. J., Rooney, E. M., Rogerson, J. H., McQuater, R. E., Spring, M., & Dale, B. G. (1999). New product design and development: a generic model. The TQM Magazine, 11(3), 172-179.
• Pugh, S. (1990). Total design: integrated methods for successful product engineering. England: W.
• Rambo, A. C. (2011). O impacto da certificação na qualidade do setor aeroespacial brasileiro na perspectiva das organizações certificadas (Dissertação de mestrado). Universidade de Taubaté, Taubaté.
• Ramaswamy, R. (1996). Design and management of service processes. Keeping custumers for life. Engineering Process Improvemet Series. Massachusetts: Addison-Wesley Publishing Company.
• Rosende, R. C., & Pendone, L. (2016). PROSUB - Programa de desenvolvimento de submarinos: contribuições para consolidação da base industrial de defesa marítima brasileira. Revista da Escola de Guerra Naval, 22(1), 33-68.
• Rosenthal, S. R. (1992). Effective product design and development. How to cut lead time and increase custumer satisfaction. Illinois: BOI.
• Salgado, E. G., & Sampaio, P. A. (2013). A certificação ISO 9001 no continente americano: análise estatística. In Anais do 33º Encontro Nacional de Engenharia de Produção Rio de Janeiro: Associação Brasileira de Engenharia de Produção. Retrieved in 2018, November 16, from http://www.abepro.org.br/biblioteca/enegep2013_TN_STO_178_016_22775.pdf
  » http://www.abepro.org.br/biblioteca/enegep2013_TN_STO_178_016_22775.pdf
• Santiago, D. G. (2014). Estudo da estrutura organizacional do ministério da defesa para a gestão de processos de teste e avaliação conjunta de produtos de defesa: um diagnóstico e sugestões (Trabalho de conclusão de curso). Escola de Comando e Estado-Maior do Exército, Rio de Janeiro.
• Santos, J. E., & Muniz, J., Jr. (2011). Avaliação da conformidade no ministério da defesa: proposta de um modelo de sistema e de uma estrutura organizacional de coordenação. In Anais do XIV Simpósio de Administração da Produção, Logística e Operações Internacionais São Paulo: SIMPOI.
• Scielo. (2018). Busca em todos os indices: (modelo) AND (projeto de serviços) Retrieved in 2018, November 17, from https://search.scielo.org/?q=%28modelo%29+AND+%28projeto+de+servi%C3%A7os%29⟨=pt&count=50&from=16&output=site&sort=&format=summary&fb=&page=1
  » https://search.scielo.org/?q=%28modelo%29+AND+%28projeto+de+servi%C3%A7os%29⟨=pt&count=50&from=16&output=site&sort=&format=summary&fb=&page=1
• Scheuing, E. E., & Johnson, E. M. (1989). A proposed model for new service development. Journal of Services Marketing, 3(2), 25-32.
• Seddon, J. (2000). The case against ISO 9000 (pp. 1-18). Dublin: Oak Tree Pr.
• Silva, A. M., & Melo, R. M. (2017). Uma abordagem multicritério para a seleção de serviços de consultoria e certificação de Sistemas de Gestão da Qualidade. Gestão & Produção, 25(1), 160-174. http://dx.doi.org/10.1590/0104-530x2753-16
  » http://dx.doi.org/10.1590/0104-530x2753-16
• Slack, N., Chambers, S., Harland, C., Harrison, A., & Johnston, R. (2009). Administração da produção São Paulo: Atlas.
• Soratto da Silva, A. N. R. (2011). Auditorias de sistemas de gestão: competências para agregação de valor (Tese de doutorado). Universidade Federal de Santa Catarina, Santa Catarina.
• Souza, L. P. (2010). Análise crítica do processo de auditoria de sistema de gestão da qualidade no setor aeroespacial (Dissertação de mestrado). Universidade de Taubaté. Taubaté.
• Souza, L. P. (2013). Estudo sobre avaliação da conformidade de produtos de defesa (Dissertação de mestrado). Instituto Nacional de Metrologia, Qualidade e Tecnologia, Duque de Caxias.
• Tax, S. S., & Stuart, I. (1997). Designing and implementing new services: the challenges of integrating service systems. Journal of Retailing, 73(1), 105-134.
• Vincent, G. (1989). Managing new-product development. Van Nostrand Reinhold, New York.
• Watton, H. B. (1969). New-product planning. A practical guide for diversification Englewood Cliffs, NJ: Prentice-Hall.
• Wheelwright, S. C., & Clark, K. B. (1992). Revolutionizing product development. Quantum leaps in speed, efficiency and quality. New York: The Free Press.

Publication Dates

• Publication in this collection
  21 Sept 2020
• Date of issue
  2020

History

• Received
  18 Mar 2018
• Accepted
  09 Jan 2019