SciELO - Scientific Electronic Library Online

vol.40 issue2Collagen production from chicken keel bone using acid and enzymatic treatment at a temperature of 30 °CQuality pattern evaluation of frozen soursop pulps: an assessment based on chemical composition and chemometric analysis author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand




Related links


Food Science and Technology

Print version ISSN 0101-2061On-line version ISSN 1678-457X

Food Sci. Technol vol.40 no.2 Campinas Apr./June 2020  Epub Nov 25, 2019 

Original Article

Characterization of Lactobacillus plantarum strains isolated from Turkish pastırma and possibility to use of food industry

Emine DINCER1  *

Merih KIVANC2 

1Department of Nutrition and Dietetics, Faculty of Health Science, Sivas Cumhuriyet University, Sivas, Turkey

2Department of Biology, Faculty of Sciences, Eskisehir Teknik University, Eskisehir, Turkey


Five Lactobacillus plantarum strains (which were previously isolated from Turkish pastırma), were evaluated for the possibilities of using in the food industry. Antimicrobial properties, metabolic activities, exopolysaccharide production and antibiotic resistance were investigated. It was also assessed probiotics features such as adhesion ability, survival in simulated gastric environment and acid tolerance analysis. Antimicrobial activity was characterized based on effects of several enzymes, different temperature and pH. All strains showed remarkable antimicrobial activity and antimicrobial substances were found resistant to enzymes, high temperature and low pH. Metabolic activities and exopolysaccharide production of the strains were found relatively weak. Determinate to antibiotic susceptibility patterns 14 different antibiotics used and strains were found reliable in terms of transferable resistance genes except for erythromycin. All strains showed resistance to acidic condition and gastric environment. However, adhesion ability of strains was found relatively low. All findings showed that, these strains potential candidate for use in food industry especially as natural food preservatives because of antimicrobial activity capacity and one strain (S2) have remarkable potential to use as starter cultures or probiotics.

Keywords:  antimicrobial activity; lactic acid bacteria; Lactobacillus plantarum; probiotic; starter culture

1 Introduction

Traditional dry-cured meat product ‘pastırma’ is produced whole muscle obtained from beef and water buffalo. Pastırma is usually consumed at breakfast without cooking and very popular in Turkey, but it is still produced traditionally and does not have any standard of production. Production process basically compromises following steps; dry curing (salt and nitrate used as the curing agent), washing (to remove excess salt), first drying (air dried, around 15 °C), first pressing (cold pressing), second drying (air dried, around 15-20 °C), second pressing (hot pressing) and paste seasoning (the outside of the product covered with a paste called çemen). Nationally some studies have been conducted associated with textural, chemical and microbiological properties of pastırma. Data in literature demonstrated that characteristics of pastırma differ depending on the origin of muscle and it is not a suitable growth medium for many microorganisms due to its low water activity and moisture. However, as a results of studies, lactic acid bacteria (LAB), catalase positive cocci and yeasts were found resistant and able to survive during production (Kaban, 2009, 2013; Kilic, 2009; Ozturk, 2015).

LAB species are naturally found in many foods including meat products and has a particular interest by food industries due to their technological properties. There is a long tradition of using LAB for food fermentations and these bacterial group have been extensively studied different perspectives (Hurtado et al., 2012; Grosu-Tudor et al., 2014). LAB cause some changes in flavor-texture of meat products and ability to utilize sugars and other nutrients. They prevent the growth of some pathogenic microorganism by antimicrobial substance production and contribute to preservation of foods. Today, LAB strains have been widely used as starter cultures in food production to improve foods appearance, smell and taste or to prolong its durability (Kilic, 2009; Blana et al., 2014; Rzepkowska et al., 2017).

Currently, another important research area is the production of functional foods and probiotics (Champagne et al., 2018). Food and Agriculture Organization (2002) defined probiotics as ‘live microorganism when administered in adequate amounts contribute a health benefit on the host’. A successful probiotic must have some characteristics such as resistance to the acidic environment of the stomach and to bile salts of the small intestine, antimicrobial activity against important pathogens and also the capacity to adhere the intestine (De Vries et al., 2006). In 2013, The International Scientific Association for Probiotics and Prebiotics (ISAPP) organized the meeting to re-examine the concept of probiotics and agreed that the FAO/WHO definition for probiotics was still relevant but advised a minor grammatical correction: “live microorganisms that, when administered in adequate amounts, confer a health benefit on the host. The panel discussed whether certain microbial products fit under the framework of ‘probiotic’. Live cultures traditionally associated with fermented foods, were determined to be outside the framework of probiotic if they were undefined and if there were no proven health benefits associated with them (Hill., et al., 2014). LAB constitute a significant proportion of probiotic cultures used in developed countries and dairy matrices is the most valuable probiotic carrier (Maganha et al., 2014; Barat & Ozcan, 2018; Temiz & Çakmak, 2018). Among the LAB, Lactobacillus plantarum has a long history of natural occurrence and safe use in a variety of food products. It is found not only dairy, meat and vegetable products but also human gastrointestinal tract. Therefore, these species widely used for probiotic research (De Vries et al., 2006; Jabbari et al., 2017). Each strain within LAB species exhibits unique properties with according to metabolism and important characteristics of a starter or probiotic cultures are strain dependent. So that, studies which is the screening LAB strains of different origin to find new probiotic or starter culture are increasing day by day (Barbosa et al., 2015; Rzepkowska et al., 2017).

The purpose of the present work was to characterize L. plantarum strains isolated from Turkish pastırma and try to find new suitable strains that can be used in fermented meat product as starter cultures or probiotic. To our knowledge, this is the first study for use the pastırma as a source for isolation of L. plantarum.

2 Materials and methods

2.1 Bacterial strains and growth condition

L. plantarum strains were isolated from different Turkish pastırma samples and identified as described previously (Dincer & Kivanc, 2012). Indicator bacteria were obtained from the USDA Agriculture Research Service, IL. USA and our laboratory culture collection. Cultures were maintained at – 80 °C in 20% glycerol (w v-1). Prior to experimental studies cultures were melted at room temperature and L. plantarum strains were cultivated in de Man Rogosa Sharpe (MRS) broth (Merck, Turkey) 30 °C for 24 h and indicator bacterial strains were cultivated in Brain Hearth Infusion (BHI) broth (Merck, Turkey) 37 °C for 24 h.

2.2 Antimicrobial activity of strains

For a wide range of scan, indicator microorganisms were selected among the different, important food borne pathogens and LAB species, totally 16 species were used as indicator microorganisms for the evaluation of antimicrobial activities (Table 1).

Table 1 The list of indicator microorganisms used determine to antimicrobial activity. 

Indicator microorganism culture collection and source
Indicator microorganisms Source of reference
Bacillus cereus NRRL B–3711 NRRL
Bacillus subtilis NRLL B–744 NRRL
Escherichia coli NRRL B–3704 NRRL
Proteus vulgaris NRRL B–123 NRRL
Salmonella typhimurium NRRL B–4420 NRRL
Listeria monocytogenes ATCC–7644 ATCC
Pseudomonas aeruginosa ATCC 27853 ATCC
Staphylococcus aureus ATCC 6538 ATCC
Enterococcus feacalis ATCC 29212 ATCC
Yersinia enterocolitica Laboratory collection
Klebsiella pneumoniae Laboratory collection
Lactobacillus plantarum NRRL B–4496 NRRL
Lactobacillus buchneri NRRL B–1837 NRRL
Lactobacillus bulgaricus NRRL B–548 NRRL
Leuconostoc paramesenteroides Laboratory collection
Lactococcus lactis Laboratory collection

NRRL, Northern Regional Research Laboratory, USA; ATCC, American Type Culture Collection, USA; Laboratory collection, Bacteria collection, Microbiology Laboratory, Anadolu University, Eskisehir, Turkey.

Antimicrobial activity of strains were determined by agar well diffusion method (Tagg & McGiven, 1971). In this study, it was tested cell free supernatant (CFS) of strains. For this purpose, CFS was obtained as follows; strains were activated in MRS broth 48 h at 30 °C, then 80 mL MRS broth was inoculated 1% active cultures and were grown 30 °C for one night. Cells were removed by centrifugation (11.000 g 30 min), CFS were adjusted pH 6.0 ± 0.2 and frozen at -80 °C for one night, next day frozen CFS concentrated by lyophilization during two days. After lyophilization process, concentrated CFS was re-suspension in 8 mL sterile distillated water, so that the CFS samples was concentrated 10 folds and then filter-sterilized using pore size of 0.2 µm (Bennik et al., 1997). Antimicrobial activity ( × ) was calculated by Equation 1:

×=inhibition zone diameterwell diameter (1)

Afterwards the antimicrobial activity was determined, to determine whether the antimicrobial activity based on the production of hydrogen peroxide, 5µg mL-1 catalase enzyme (Sigma-Aldrich, Turkey) was added to CFS, solution incubated 4 h at 37 °C and the test was repeated. Lastly, to confirm antimicrobial activity originated from a compound which have protein nature, 1 mg mL-1 final concentration proteinase K (Sigma-Aldrich, Turkey) was dissolved 0.05 M sodium phosphate buffer, pH: 7.5 and added CFS, solutions were incubated 4 h at 37 °C and the test was repeated (Zhu et al., 2000).

2.3 Characterization of antimicrobial activity

Considering the results of antimicrobial activity assay, only selected 4 indicator microorganism were used to characterization of antimicrobial activity. Enterococcus feacalis ATCC-29212, Listeria monocytogenes ATCC–7644, and Staphylococcus aureus ATCC-6538 were selected to represent Gram positive food borne pathogens. Pseudomonas aeroginosa ATCC-27853 were selected to represent Gram negative food borne pathogens. Antimicrobial activity was characterized based on effects of several enzymes, different temperature and pH.

The following enzymes were used for this work: trypsin (2 mg mL-1), α-chymotrypsin (5 mg mL-1), α-amylase (1 mg mL-1), lysozyme (1 mg mL-1) and pronase (1 mg mL-1). All enzymes were purchased from Sigma-Aldrich, Turkey and dissolved 0.05 M sodium phosphate buffer, pH: 7.5. One by one each enzyme was added CFS of strains, then the solutions were incubated 4 h at 37 °C and right after remaining antimicrobial activity was determined as previously, by agar well diffusion protocols (Zhu et al., 2000).

Because of determine the temperature sensitivity, CFS samples were prepared from each strain and divided into eight pieces. Each piece was exposed to a certain degree of temperature for a certain period. Temperature degree and periods were used as follows, 30 min at 50, 60, 70, 80, 90, 100, 110 °C and 20 min at 120 °C. At the end of this periods remaining antimicrobial activity was detected again with the treated CFS samples by agar well diffusion protocols.

In order to determine the effect of pH, CFS samples were prepared and CFS from each strains were divided into seven piece and adjusted to pH (pH: 1, 3, 5, 7, 9, 11, and 13). Afterward this samples were incubated 24 h at 37 °C, then pH was adjusted again 6.0 ± 0.2 and antimicrobial activity was determined, as previously (Bhunia et al., 1988; Zhu et al., 2000).

2.4 Determination of metabolic products and EPS production

Proteolytic activity and amounts of hydrogen peroxide (H2O2) was detected with regard to Rajagopal & Sandine (1990) and Patrick & Wagner (1949), respectively. Proteolytic activity and H2O2 are expressed as mg tyrosine mL-1and µg H2O2 mL-1. To estimate the amounts of lactic acid, measurements were carried out spectrophotometrically at 400 nm according to the Demirci & Gunduz (1994). The amount of lactic acid is expressed as mg lactic acid mL-1.

EPS production capabilities were carried out modified MRS agar medium. To pre-scanning, 4 type modified MRS agar was prepared with the same MRS content but different carbon source (glucose, lactose, fructose, sucrose) and each strain was growth 24-48 h at 30 °C. Then, cultures which have ropy appearance and mucoid structure were selected and inoculated in MRS broth including same carbon source and incubated 24-48 h at 30 °C. After that this cultures were inoculated by 1% in same broth medium and incubated 18 °C. During the incubation period, 48 and 72 h after, viscosity of samples was measured by low scale viscosity meter (Thermo HAAKE Viscositer 6 plus). Measurement was carried out 3 different revolutions including 200 rpm, 100 rpm and 60 rpm. Sterile inoculation MRS broth medium was used as standards during the measurements (Ruas-Madiedo & De los Reyes-Gavilán, 2005; Vijayendra et al., 2008).

2.5 Safety assessment-antibiotic resistance of strains

Antibiotic resistance patterns of strains were determined by using agar disk diffusion method as described firstly by Bauer et al. (1966). All antibiotic discs were purchased from Oxoid-Hemakim, Turkey. To scan a wide range of determinants, antibiotics was chosen from different antibiotic groups in the form of β–lactams, aminoglycosides, fluoroquinolones, macrolides, broad spectrum, cephalosporin and glycopeptides. The choice of antibiotic concentrations, analysis procedure and evaluation of results was determined accordance with the guidelines proposed by the Clinical and Laboratory Standards Institute (2010).

2.6 Assessment of probiotic features

In order to determine resistance under acidic conditions, strains incubated in the MRS broth for 18 h at 30 °C were harvest by centrifugation (10.000 g 10 min), washed twice in PBS and cell density of adjusted to McFarland No: 0.5 standards (bioMe´rieux, Marcy l’Etoile, France). Then, 1000 µL of these cultures were inoculated 9 mL MRS broth adjusted to pH 2.5. The numbers of viable bacteria were determined by plate counting on MRS agar after exposure to acidic condition for 0, 3 and 6 hours at 37 °C. Plates were incubated 48 h at 30 °C and survival cell count were expressed as log values of CFU mL-1 (Thirabunyanon et al., 2009).

Survival of strains in gastric environment was determined according to the methods of Corcoran et al. (2005). Briefly, strains were grown in MRS broth, centrifuged at 7,000 g 10 min and washed once in ringer solution. Pellet was re-suspended in simulated gastric juice at 37 °C. The numbers of viable bacteria were determined by plate counting on MRS agar after incubation 0, 10, 30, 60, 90 min at 37 °C. Plates were incubated for 48 h at 30 °C and survival cell count were expressed as log values of CFU mL-1. Simulated gastric juice was formulated using glucose (3.5 g L-1), NaCl (2.05 g L-1), KH2PO4 (0.60 g L-1), CaCl2 (0.11 g L-1) and KCl (0.37 g L-1), adjusted to pH 2.0 using 1 M HCl, and autoclaved at 121 °C for 15 min. Porcine bile (0.05 g L-1), lysozyme (0.1 g L-1), and pepsin (13.3 mg L-1) were added as stock solutions prior to analysis.

Adhesion properties of strains were evaluated using 106 Caco-2 cells well-1 in 6 well tissue culture plates. Human colon adenocarcinoma, Caco-2 cell line (Accession Number: 98052301) were purchased Republic of Turkey Ministry of Food Agriculture and Livestock, Foot & Mouth Disease Institute. Strains in MRS broth incubated for 18 h at 30 °C were harvested and washed twice with PBS and re-suspended in non-supplemented Dulbecco’s Modified Eagle’s Medium (DMEM) to adjust 108 CFU mL-1. After washing the Caco-2 twice with PBS, 0.5 mL bacterial suspension was added to each well and incubated for 1 h at 37 °C in 5% CO2. Unattached bacteria were removed by washing with PBS three times. Caco-2 cells were lysed with 0.1% (v v-1) Triton X-100 for 5 min at 37 °C and lysates were serially diluted and plated on MRS agar. Plates were incubated for 48 h at 30 °C and attached bacterial cells count were expressed as log values of CFU mL-1 (Thirabunyanon et al., 2009). Adherence percentage was calculated by Equation 2:

% Adhesion=Final count of strainsCFU mL1Initial number of strainsCFU mL1×100 (2)

3 Results and discussion

While all strains showed high antimicrobial activity against most of the indicator microorganisms (Table 2), any strain showed effect against indicator LAB. Because of all strains ineffective against indicator LAB, these indicator organisms not shown in Table 2. Antimicrobial activity of potential starter cultures or probiotic organisms is one of the important feature for selection criteria. Antimicrobial activity can be due to production of bacteriocin or other antimicrobial substance (Piard & Desmazeaud, 1992). As shown in Table 2, strains that exhibit significant antimicrobial activity against indicator organisms.

Table 2 Antimicrobial activity of cell-free supernatants (CFS), catalase added CFS and proteinase K added CFS of tested LAB against foodborne pathogens. Values represent the average of two independent experiments and standard deviation from two replications. 

L. plantarum strains / treatment The average diameter of growth inhibition zones observed for tested indicator microorganisms [mm]
P. vulgaris B. cereus E. coli B. subtilis K. pneumoniae S. typhimurium E. feacalis Y. enterocolitica S. aureus L. monocytogenes P. aeruginosa
S1 A - - 5.3 ± 0.12 - 4.2 ± 0.00 - 4.6 ± 0.25 4.1 ± 0.20 4.1 ± 0.35 4.0 ± 0.55 6.5 ± 0.18
B - - 5.0 ± 0.39 - 4.2 ± 0.00 - 4.4 ± 0.40 3.9 ± 0.55 4.1 ± 0.28 4.0 ± 0.44 6.2 ± 0.00
C - - - - - - - 3.1 ± 018 - - -
S2 A 4.6 ± 0.30 8.2 ± 0.10 4.5 ± 0.25 4.8 ± 0.22 6.3 ± 0.15 6.6 ± 0.25 7.1 ± 0.20 6.0 ± 0.00 5.2 ± 0.25 5.5 ± 0.00 6.3 ± 0.50
B 4.4 ± 0.21 2.1 ± 0.55 4.5 ± 0.13 4.7 ± 0.35 6.0 ± 0.00 4.9 ± 0.55 4.8 ± 0.35 - 4.3 ± 0.44 5.1 ± 0.12 6.1 ± 0.55
C - - 4.5 ± 0.10 - - 4.9 ± 0.50 - - - - -
S3 A 8.3 ± 0.25 8.1 ± 0.45 8.0 ± 0.22 6.4 ± 0.37 8.5 ± 0.00 9.2 ± 0.28 6.3 ± 0.82 6.7 ± 0.10 8.4 ± 0.35 8.8 ± 0.18 6.7 ± 0.27
B 8.3 ± 0.15 2.7 ± 0.00 8.0 ± 0.10 - 8.5 ± 0.00 5.6 ± 0.72 6.0 ± 0.46 - 8.2 ± 0.20 8.3 ± 0.50 6.1 ± 0.62
C - - 3.0 ± 0.15 - - 4.8 ± 0.30 - - - - -
S4 A 5.3 ± 0.28 7.1 ± 1.25 7.2 ± 0.00 - 9.1 ± 0.55 6.7 ± 0.50 8.3 ± 0.55 7.2 ± 0.40 8.1 ± 0.68 8.1 ± 0.25 8.3 ± 0.62
B 5.2± 0.32 3.9 ± 0.47 4.5 ± 0.55 - 8.1± 0.51 4.3 ± 0.72 6.9 ± 0.62 7.2 ± 0.44 2.8 ± 0.00 7.6 ± 0.55 8.3 ± 0.48
C - - - - - - - - - - -
S5 A 4.2 ± 0.55 6.3 ± 0.00 8.0 ± 0.52 8.4 ± 0.55 8.2 ± 0.78 7.6 ± 0.55 4.1 ± 0.51 8.3 ± 0.50 6.2 ± 0.81 7.1 ± 0.22 4.0 ± 0.55
B 4.2 ± 0.32 4.5 ± 0.10 7.3 ± 0.12 6.1 ± 0.75 6.8 ± 0.25 4.9 ± 0.35 4.1 ± 0.10 - 6.0 ± 0.55 7.1 ± 0.95 4.0 ± 0.00
C - - - - - - - - - - -

A, cell free supernatant (CFS); B, catalase added CFS; C, proteinase K added CFS; (-), no inhibition zones.

Other researchers have also demonstrated that antimicrobial activity of LAB strains against foodborne pathogens, nevertheless this activity is usually restricted gram positive bacteria (Prudêncio et al., 2015; Arena et al., 2016; Jabbari et al., 2017; Rzepkowska et al., 2017). In this respect, our profiling of antimicrobial activity against gram negative bacteria is notable. CFS of strains were adjusted pH 6.0 ± 0.2, therefore were known to antimicrobial activity isn’t derives from acidity. Also, as indicated that in Table 2, after the catalase treatment, the absence of a decrease or small reduction in antimicrobial activity showed that it isn’t derives from hydrogen peroxide production. On the contrary, antimicrobial activities of all strains were almost completely inactivated after the Proteinase K addition. These two data indicate that; antimicrobial activities of strains could be due to the presence of bacteriocin or bacteriocin like metabolites.

Because of the protein structure, bacteriocins are generally affected by proteolytic enzymes, temperature or pH (Piard & Desmazeaud, 1992). For these reasons, antimicrobial activity decrease had been considered as expected result with treated CFS added proteinase K. Antimicrobial activity changes by the effect of proteolytic enzymes depend on used strain and indicator pathogen microorganism and results are presented in Table 3. After the treatment of proteolytic enzymes, antimicrobial activity was partially decreased. However, there were found any enzyme which eliminates the antimicrobial activity against the entire indicator organism. For all that, even after treatment with the proteolytic enzymes, maintained antimicrobial activity of strains are a remarkable result.

Table 3 Antimicrobial activity change by the effect of proteolytic enzymes and pH. 

L. plantarum strains and treatment The average diameter of growth inhibition zones observed for tested indicator microorganisms [mm]
L. monocytogenes E. feacalis S. aureus P. aeruginosa
S1 Trypsin 4.0 ± 0.25 4.3 ± 0.18 3.0 ± 0.65 6.2 ± 0.22
α-chymotrypsin 2.4 ± 0.38 4.1 ± 0.50 3.0 ± 0.41 6.0 ± 0.27
Lysozyme - 4.1 ± 0.44 3.4 ± 0.13 6.2 ± 0.15
α-amylase 3.1 ± 0.17 3.4 ± 0.28 - -
Pronase - 4.0 ± 0.55 - 2.6 ± 0.32
pH 3 2.7 ± 0.39 - - 3.3 ± 0.18
pH 5 4.0 ±0.00 4.3 ± 0.20 4.1 ± 0.23 6.5 ± 0.10
pH 7 4.0 ± 0.43 - - 2.5 ± 0.55
S2 Trypsin 1.3 ± 0.55 - 1.6 ± 0.25 1.9 ± 0.42
α-chymotrypsin 2.1 ± 0.48 7.0 ± 0.35 - 6.2 ± 0.61
Lysozyme 5.5 ± 0.00 4.4 ± 0.20 1.5 ± 0.17 6.2 ± 0.15
α-amylase 5.5 ± 0.00 - 2.2 ± 0.34 6.1 ± 0.59
Pronase 2.4 ± 0.18 2.0 ± 0.25 - 4.3 ± 0.28
pH 3 - 4.1 ±0.00 - 4.0 ± 0.41
pH 5 5.3 ± 0.00 7.0 ± 0.55 - 6.2 ± 0.49
pH 7 2.6 ± 0.10 2.5 ± 0.15 - -
S3 Trypsin 5.3 ± 0.65 3.2 ± 0.42 4.5 ± 0.25 6.0 ± 0.50
α-chymotrypsin - 3.2 ± 0.37 4.1 ± 0.00 6.0 ± 0.48
Lysozyme - 3.0 ± 0.60 2.4 ± 0.19 6.1 ± 0.15
α-amylase 2.8 ± 0.10 3.2 ± 0.45 2.0 ± 0.44 -
Pronase - 3.0 ± 0.50 - 3.1 ± 0.55
pH 3 - - - -
pH 5 4.9 ± 0.50 6.2 ± 0.00 - 4.3 ± .020
pH 7 - - - -
S4 Trypsin 4.7 ± 0.50 8.1 ± 0.52 4.5 ± 0.00 6.1 ± 0.15
α-chymotrypsin 2.2 ± 0.38 4.6 ± 0.28 4.1 ± 0.10 6.0 ± 0.00
Lysozyme 8.0 ± 0.20 7.7 ± 0.32 7.3 ± 0.15 8.1 ± 0.55
α-amylase 3.1 ± 0.42 8.3 ± 0.50 3.0 ± 0.10 1.0 ± 0.00
Pronase - 4.9 ± 0.23 - 2.6 ± 0.38
pH 3 - 4.3 ± 0.55 - 4.2 ± 0.00
pH 5 6.5 ± 0.13 4.1 ± 0.18 - 3.9 ± 0.55
pH 7 4.8 ± 0.00 2.7 ± 0.35 - 1.7 ± 0.27
S5 Trypsin 5.2 ± 0.36 4.0 ± 0.55 5.2 ± 0.28 2.3 ± 0.71
α-chymotrypsin 3.5 ± 0.18 4.0 ± 0.23 5.0 ± 0.47 2.1 ± 0.50
Lysozyme 3.4 ± 0.55 4.1 ± 0.15 4.9 ± 0.62 2.2 ± 0.17
α-amylase - 4.0 ± 0.55 1.0 ± 0.00 -
Pronase - 4.1 ± 0.25 1.1 ± 0.10 2.8 ± 0.00
pH 3 - - - -
pH 5 1.8 ± 0.39 4.1 ± 0.15 - 2.4 ± 0.87
pH 7 - - - -

All of the strains were found to be resistant to heat change. Even if strains exposed to 120 °C for 20 min, the antimicrobial activity of strains was not change. After the CFS samples were incubated different pH value, were observed to completely lose the antimicrobial activity of the all strains at pH 1, 9, 11 and 13. Therefore this results are not given in Table 3. The closest antimicrobial activity to the original was found at pH 5. When the strains were evaluated individually, S1 and S2 were found to be less affected strains by change in the pH. These strains and S4 showed antimicrobial activity at pH 3, 5 and 7, S3 and S5 showed antimicrobial activity only pH 5, results are presented in Table 3. Bacteriocins from LAB to be divided into four main classes (class I, class II, class III and class IV) and some subclasses. Among them, subclass IIa bacteriocins are (pediocin like bacteriocin) active against gram-positive food spoilage and pathogenic bacteria and in general pediocin like bacteriocins are considered stable against pH and temperature change. For this reason subclass IIa bacteriocins can be pointed out as important groups for use in food preservation. Our antimicrobial activity results (broad inhibitory spectrum, resistant to proteolytic enzymes, high temperature and low pH) indicated that antimicrobial substance produced by our strains within the subclass IIa (Drider et al., 2006).

LAB produce various antimicrobial substances such as lactic acid and hydrogen peroxide. Lactic acid is the major metabolic last product of carbohydrate fermentation and it is a commercially valuable product to use for food manufacturing and pharmaceutical industries. Hydrogen peroxide is another important metabolic product which was produced by some Lactobacillus species and it is deemed beneficial for food preservation (Zhu et al., 2000; Yuksekdag & Aslım, 2010). Furthermore, some LAB strains provide the formation of free amino acid and small peptides during proteolytic activities. These compounds may also be formed contribute formation of flavor in some foods, so LAB indirectly play important role the formation of flavor in some foods (Law & Haandrikman, 1997).

In this work, all strains found to be a manufacturer of lactic acid and hydrogen peroxide, besides this, only three strains were exhibited high proteolytic activity. The results of metabolic product analysis are presented Table 4. According to the results, amount of lactic acid, proteolytic activity and hydrogen peroxide produced by the strains found ranged between 10.31–21.37 mg mL-1, 0.008–0.445 mg mL-1 and 0.991–1.044 µg mL-1, respectively.

Table 4 Amounts of metabolic products were produced by strains. Values represent the average of three independent experiments and standard deviation from three replications. 

The average amounts of metabolic products observed for tested LAB strain
S1 S2 S3 S4 S5
pH 4.29 3.93 4.21 4.64 4.26
Lactic acid (mg/mL) 10.31 ± 0.006 19.74 ± 0.001 13.99 ± 0.004 21.37 ± 0.012 15.65 ± 0.003
H2O2 (µg/mL) 0.991 ± 0.039 1.044 ± 0.091 1.037 ± 0.014 1.055 ± 0.033 1.001 ± 0.047
Proteolytic Activity (mg/mL) 0.122 ± 0.040 0.445 ± 0.072 0.003 ± 0.012 0.008 ± 0.010 0.440 ± 0.031
EPS production G (+) - - - S (+)
S (+)
Strain Sugar 48 hour 72 hour
Viscosity Accuracy % Viscosity Accuracy %
S1 Sucrose 2.03 72 2 70
S1 Glucose 2 71 2 70
S5 Sucrose 2.48 87 2.48 86

G (+), positive EPS production in the medium containing glucose; S (+), positive EPS production in medium containing sucrose; (-), no EPS production.

In the food industry, EPS produced by LAB are used to modify rheological properties and texture of product. Therefore, EPS production is advantageous feature but it is not mandatory for use as a starter or probiotic culture (Ruas-Madiedo & De los Reyes-Gavilán, 2005). EPS production is widespread in LAB, however all strains used in this study was found as weak EPS producer (Table 4). Only S1 and S5 strains were found as EPS producer in medium containing glucose and sucrose, respectively. On the other hand, viscosity of these two strain was found to be low according to the measurements carried out with low scale viscosity meter.

Antibiotic resistance/susceptibility patterns of the strains are presented in Table 5. All strains found that were resistant to streptomycin and ceftriaxone. The findings showed that, all strains except of S1 have resistant kanamycin, amikacin and erythromycin. All strains showed sensitivity to a similar extent against chloramphenicol, tetracycline, moxifloxacin, ciprofloxacin and gatifloxacin antibiotics. In this study, only one vancomycin resistant strain was found. Potential probiotic LAB must be safe for human consumption and they should not show acquired or transferable antibiotic resistance. If antibiotic resistance genes are present on plasmids, they could be transferred to other bacteria (Vizoso Pinto et al., 2006). Lactobacillus species are generally have natural resistant to aminoglycosides and inhibitors of nucleic acid synthesis but susceptible to antibiotics which inhibit the protein synthesis and β-lactamase inhibitors (Ammor et al., 2007) and our findings are in the same direction. Similarly, Vizoso Pinto et al. (2006) reported that seven selected Lactobacillus strains resistant to streptomycin, gentamicin, ciprofloxacin and it is susceptible erythromycin, penicillin and chloramphenicol. The results of another study conducting 23 potential probiotics Lactobacillus strain showed that strains resistance towards gentamicin, ciprofloxacin, kanamycin and streptomycin (Mathara et al., 2008). Compared with the literature data, our results generally were similar with other authors’ results. Among the antibiotics resistances, vancomycin, tetracycline and erythromycin resistance is major concern because these have been shown to be transferable by Lactobacillus (Ammor et al., 2007; Mathara et al., 2008). In this regard especially vancomycin and tetracycline susceptibility of our strains was interpreted as positive result. After all, it may be accepted that the strains used in this study is reliable in terms of transferable resistant to antibiotics. But our strains were found resistant to erythromycin. Erythromycin resistance must be investigated with molecular methods and guaranteed not show the transferable resistance.

Table 5 Antibiotics resistance / susceptibility patterns of strains. 

Type of antibiotic Tested LAB strains
Groups Antibiotics µg/disc S1 S2 S3 S4 S5
Aminoglycosides Gentamicin (CN) -10 µg S S R S R
Netilmicinsulphate (NET) - 30 µg S S R S R
Kanamycin (K) - 30 µg I R R R R
Streptomycin (SH) - 10 µg R R R R R
Amikacin (AK) - 30 µg I R R R R
Broad spectrum Chloramphenicol (C) - 30 µg S S S S S
Tetracycline (TE) – 30 µg S S S S S
Fluoroquinolones Lomefloxacin (LOM) - 10 µg R R R R R
Ciprofloxacin (CIP) - 5 µg I S I S I
Gatifloxacin (GAT) - 5 µg S S S S S
Cephalosporin Ceftriaxone (CRO) - 30 µg R R R R R
Macrolides Erythromycin (E) - 10 µg S R R R R
β–lactams Penicillin G - 5 U S S R S S
Glycopeptides Vancomycin (VA) - 30 µg R S S S S

R, resistant; S, susceptible; I, Intermediate.

In order to be accepted as a probiotic of a microorganism, that organism has to ability of adherence to human intestinal epithelial cells. Therefore, Caco-2 cell line has been used typically as an in vitro model. Although, adhesion capacity of human intestinal cell is strain specific characteristic, adhesion rates of lots of strains had to low in many studies (Ramos et al., 2013; Thirabunyanon et al., 2009). In our study, all strains have low adhesion to Caco-2 cell (Figure 1). Among the tested strains, only S1 and S2 have more than 1% adhesion, other three strains were considered not to have adhesion ability to Caco-2 cells and this case restrict the possibilities of strains are used as probiotics. Similarly, Lim & Im (2009) reported that 4 adhesive strain and 117 strains had low adhesion capacity. Maragkoudakis et al. (2006) reported that 20 out of 29 tested strain less than 4% adhesion.

Figure 1 Adhesion ability of strains to CaCo-2 cells. Values represent the average of three independent experiments and error bars indicate standard deviation from three replications per strain. 

Other properties to use of probiotic, must be that microorganism is resistant to acidic condition and survive to gastric environment. With this feature, microorganism reaches the small intestine, becomes colonized and shows beneficial effects on the host. (Corcoran et al., 2005). In our study, with a small amount of viability loss, all strains were found resistant to pH 2.5 after 3 h of exposure (Figure 2). However, after exposure to acidic condition for 6 h at 37 °C, all strains were lose completely viability (data not shown). All strains have high resistant to gastric environment and results are presented Figure 3. The results showed that, generally there is no difference between 30 and 60 min incubation while all strains shower that different degree’s viability loss after 10 and 90 min incubation in gastric juice. Even, after that 90 min incubation, concentration of strains was found 7.0 CFU mL-1 or above, except of S1. These properties consolidated the possibility of using as probiotic of our stains.

Figure 2 Acid tolerance activity of strains. Values represent the average of three independent experiments and error bars indicate standard deviation from three replications per strain. 

Figure 3 Bacterial counts to simulated gastric environment. Values represent the average of three independent experiments and error bars indicate standard deviation from three replications per strain. 

In this study we investigated some probiotic features our strains, but further optimization studies needed to enable our strains be used as probiotics. Particularly, in vivo studies are essential for a complete definition of the probiotic status. These studies can be performed in mice or humans (Lollo et al., 2015; Moura et al., 2016; Martins et al., 2018; Sperry et al., 2018; Mostafai et al., 2019; Nadelman et al., 2019).

4 Conclusion

Main objective of this study was to describe the important characteristics of L. plantarum strains isolated from Turkish pastırma and try to find new candidate strains for usage to starter cultures or probiotics. All of the five L. plantarum strains have broad inhibitory spectrum, produced antimicrobial compounds resistant to pH and temperature change, high capacity to production of metabolic products and shows the vancomycin susceptibility. Based on these data we decided that these strains could be potential candidate for using in fermented meat products. However, considering the use as probiotics, low adhesion ability restrict the possibilities of strains are used. In this study, when considering the all features, especially S2 strain promise to as a new candidate strain for starter cultures or probiotic, it could be use after optimizations studies.


This project was financially supported by ‘Anadolu University Council of Research Project Fund’ (Project No: 41020) for their support of chemicals and instruments.

Practical Application: Find new suitable lactic acid bacteria strains that can be used in fermented meat product as starter cultures or probiotic


Ammor, M. S., Belén Flórez, A., & Mayo, B. (2007). Antibiotic resistance in non-enterococcal lactic acid bacteria and bifidobacteria. Food Microbiology, 24(6), 559-570. PMid:17418306. [ Links ]

Arena, M. P., Silvain, A., Normanno, G., Grieco, F., Drider, D., Spano, G., & Fiocco, D. (2016). Use of Lactobacillus plantarum strains as a bio-control strategy against food-borne pathogenic microorganisms. Frontiers in Microbiology, 7, 464. PMid:27148172. [ Links ]

Barat, A., & Ozcan, T. (2018). Growth of probiotic bacteria and characteristics of fermented milk containing fruit matrices. International Journal of Dairy Technology, 71, 120-129. [ Links ]

Barbosa, J., Borges, S., & Teixeira, P. (2015). Pediococcus acidilactici as a potential probiotic to be used food industry. International Journal of Food Science & Technology, 50(5), 1151-1157. [ Links ]

Bauer, A. W., Kirby, W. M. M., Sherris, J. C., & Turck, M. (1966). Antibiotic susceptibility testing by a standardized single disk method. American Journal of Clinical Pathology, 45(4), 493-496. PMid:5325707. [ Links ]

Bennik, M. H., Verheul, A., Abee, T., Naaktgeboren-Stoffels, G., Gorris, L. G., & Smid, E. J. (1997). Interactions of nisin and pediocin pa-1 with closely related lactic acid bacteria that manifest over 100-fold differences in bacteriocin sensitivity. Applied and Environmental Microbiology, 63(9), 3628-3636. PMid:9293015. [ Links ]

Bhunia, A. K., Johnson, M. C., & Ray, B. (1988). Purification, characterization and antimicrobial spectrum of a bacteriocin produced by Pediococcus acidilactici. The Journal of Applied Bacteriology, 65(4), 261-268. PMid:2906056. [ Links ]

Blana, V. A., Grounta, A., Tassou, C. C., Nychas, G.-J. E., & Panagou, E. Z. (2014). Inoculated fermentation of green olives with potential probiotic Lactobacillus pentosus and Lactobacillus plantarum starter cultures isolated from industrially fermented olives. Food Microbiology, 38, 208-218. PMid:24290645. [ Links ]

Champagne, C. P., Gomes da Cruz, A., & Daga, M. (2018). Strategies to improve the functionality of probiotics in supplements and foods. Current Opinion in Food Science, 22, 160-166. [ Links ]

Clinical and Laboratory Standards Institute. (2010). M100 - Performance standards for antimicrobial susceptibility testing. Twentieth Informational, 39(1 Suppl.), 1-25. [ Links ]

Corcoran, B. M., Stanton, C., Fitzgerald, G. F., & Ross, R. P. (2005). Survival of probiotic lactobacilli in acidic environments is enhanced in the presence of metabolizable sugars. Applied and Environmental Microbiology, 71(6), 3060-3067. PMid:15933002. [ Links ]

De Vries, M. C., Vaughan, E. E., Kleerebezem, M., & De Vos, W. M. (2006). Lactobacillus plantarum—Survival, functional and potential probiotic properties in the human intestinal tract. International Dairy Journal, 16(9), 1018-1028. [ Links ]

Demirci, M., & Gunduz, H. (1994). Dairy technology handbook. Ankara: Hasad Press. [ Links ]

Dincer, E., & Kivanc, M. (2012). Characterization of lactic acid bacteria from Turkish pastirma. Annals of Microbiology, 62(3), 1155-1163. [ Links ]

Drider, D., Fimland, G., Hechard, Y., McMullen, L. M., & Prevost, H. (2006). The continuing story of Class IIa Bacteriocins. Microbiology and Molecular Biology Reviews, 70(2), 564-582. PMid:16760314. [ Links ]

Food and Agriculture Organization, World Health Organization. (2002). Guidelines for the evaluation of probiotics in food: report of a joint FAO/WHO working group on drafting guidelines for the evaluation of probiotics in food. London: FAO/WHO. [ Links ]

Grosu-Tudor, S. S., Stancu, M. M., Pelinescu, D., & Zamfir, M. (2014). Characterization of some bacteriocins produced by lactic acid bacteria isolated from fermented foods. World Journal of Microbiology & Biotechnology, 30(9), 2459-2469. PMid:24849010. [ Links ]

Hill, C., Guarner, F., Reid, G., Gibson, G. R., Merenstein, D. J., Pot, B., Morelli, L., Canani, R. B., Flint, H. J., Salminen, S., Calder, P. C., & Sanders, M. E. (2014). The International Scientific Association for Probiotics and Prebiotics consensus statement on the scope and appropriate use of the term probiotic. Nature Reviews. Gastroenterology & Hepatology, 11(8), 506-514. PMid:24912386. [ Links ]

Hurtado, A., Reguant, C., Bordons, A., & Rozes, N. (2012). Lactic acid bacteria from fermented table olives. Food Microbiology, 31(1), 1-8. PMid:22475936. [ Links ]

Jabbari, V., Khiabani, M. S., Mokarram, R. R., Hassanzadeh, A. M., Ahmadi, E., Gharenaghadeh, S., Karimi, N., & Kafil, H. S. (2017). Lactobacillus plantarum as a probiotic potential from kouzeh cheese (traditional Iranian cheese) and its antimicrobial activity. Probiotics and Antimicrobial Proteins, 9(2), 189-193. PMid:28155128. [ Links ]

Kaban, G. (2009). Changes in the composition of volatile compounds and in microbiological and physicochemical parameters during pastırma processing. Meat Science, 82(1), 17-23. PMid:20416610. [ Links ]

Kaban, G. (2013). Sucuk and pastırma: Microbiological changes and formation of volatile compounds. Meat Science, 95(4), 912-918. PMid:23608196. [ Links ]

Kilic, B. (2009). Current trends in traditional Turkish meat products and cuisine. Food Scencei Technology, 42(10), 1581-1589. [ Links ]

Law, J., & Haandrikman, A. (1997). Proteolytic enzymes of lactic acid bacteria. International Dairy Journal, 7(1), 1-11. [ Links ]

Lim, S. M., & Im, D. S. (2009). Screening and characterization of probiotic lactic acid bacteria isolated from Korean fermented foods. Journal of Microbiology and Biotechnology, 19(2), 178-186. PMid:19307768. [ Links ]

Lollo, P. C. B., Morato, P. N., Moura, C. S., Almada, C. N., Felicio, T. L., Esmerino, E. A., Barros, M. E., Amaya-Farfan, J., Sant’Ana, A. S., Raices, R. R. S., Silva, M. C., & Cruz, A. G. (2015). Hypertension parameters are attenuated by the continuous consumption of probiotic Minas cheese. Food Research International, 76(Pt 3), 611-617. PMid:28455044. [ Links ]

Maganha, L. C., Rosim, R. E., Corassin, C. H., Cruz, A. G., Faria, J. A. F., & Oliveira, C. A. F. (2014). Viability of probiotic bacteria in fermented skim milk produced with different levels of milk powder and sugar. International Journal of Dairy Technology, 67(1), 89-94. [ Links ]

Maragkoudakis, P. A., Zoumpopoulou, G., Miaris, C., Kalantzopoulos, G., Pot, B., & Tsakalidou, E. (2006). Probiotic potential of Lactobacillus strains isolated from dairy products. International Dairy Journal, 16(3), 189-199. [ Links ]

Martins, A. A., Santos-Junior, V. A., Filho, E. R. T., Silva, H. L. A., Ferreira, M. V. S., Graça, J. S., Esmerino, E. A., Lollo, P. C. B., Freitas, M. Q., Sant’Ana, A. S., Costa, L. E. O., Raices, R. S. L., Silva, M. C., Cruz, A. G., & Barros, M. E. (2018). Probiotic Prato cheese consumption attenuates development of renal calculi in animal model of urolithiasis. Journal of Functional Foods, 49, 378-383. [ Links ]

Mathara, J. M., Schillinger, U., Guigas, C., Franz, C., Kutima, P. M., Mbugua, S. K., Shin, H. K., & Holzapfel, W. H. (2008). Functional characteristics of Lactobacillus spp. from traditional maasai fermented milk products in Kenya. International Journal of Food Microbiology, 126(1-2), 57-64. PMid:18539351. [ Links ]

Mostafai, R., Nachvakc, S. M., Mohammadi, R., Rocha, R. S., Silva, M. C., Esmerino, E. A., Nascimento, K. O., Cruz, A. G., & Mortazavian, A. M (2019). Effects of vitamin D-fortified yogurt in comparison to oral vitamin D supplement on hyperlipidemia in pre-diabetic patients: a randomized clinical trial. Journal of Functional Foods, 52, 116-120. [ Links ]

Moura, C. S., Lollo, P. C. B., Morato, P. N., Esmerino, E. A., Margalho, L. P., Santos-Junior, V. A., Coimbra, P. T., Cappato, L. P., Silva, M. C., Garcia-Gomes, A. S., Granato, D., Bolini, H. M. A., Sant’Ana, A. S., Cruz, A. G., & Amaya-Farfan, J. (2016). Assessment of antioxidant activity, lipid profile, general biochemical and immune system responses of Wistar rats fed with dairy dessert containing Lactobacillus acidophilus La-5. Food Research International, 90, 275-280. PMid:29195882. [ Links ]

Nadelman, P., Monteiro, A., Balthazar, C. F., Silva, H. L. A., Cruz, A. G., Neves, A. A., Fonseca-Gonçalves, A., & Maia, L. C. (2019). Probiotic fermented sheep’s milk containing Lactobacillus casei 01: Effects on enamel mineral loss and Streptococcus counts in a dental biofilm model. Journal of Functional Foods, 54, 241-248. [ Links ]

Ozturk, I. (2015). Presence, changes and technological properties of yeast species during processing of pastirma, a Turkish dry-cured meat product. Food Control, 50, 76-84. [ Links ]

Patrick, W. A., & Wagner, H. B. (1949). Determination of hydrogen peroxide in small concentrations. Analytical Chemistry, 21(10), 1279-1289. [ Links ]

Piard, J. C., & Desmazeaud, M. (1992). Inhibiting factors produced by lactic acid bacteria. 2. Bacteriocins and other antibacterial substances. Le Lait, 72(2), 113-142. [ Links ]

Prudêncio, C. V., Santos, M. T., & Vanetti, M. C. (2015). Strategies for the use of bacteriocins in Gram-negative bacteria: relevance in food microbiology. Journal of Food Science and Technology, 52(9), 5408-5417. PMid:26344957. [ Links ]

Rajagopal, S. N., & Sandine, W. E. (1990). Associative growth and proteolysis of Streptococcus thermophilus and Lactobacillus bulgaricus in skim milk. Journal of Dairy Science, 73(4), 894-899. [ Links ]

Ramos, C. L., Thorsen, L., Schwan, R. F., & Jespersen, L. (2013). Strain-specific probiotics properties of Lactobacillus fermentum, Lactobacillus plantarum and Lactobacillus brevis isolates from Brazilian food products. Food Microbiology, 36(1), 22-29. PMid:23764216. [ Links ]

Ruas-Madiedo, P., & De los Reyes-Gavilán, C. G. (2005). Invited review: methods for the screening, isolation, and characterization of exopolysaccharides produced by lactic acid bacteria. Journal of Dairy Science, 88(3), 843-856. PMid:15738217. [ Links ]

Rzepkowska, A., Zielińska, D., Ołdak, A., & Kołozyn-Krajewska, D. (2017). Organic whey as a source of Lactobacillus strains with selected technological and antimicrobial properties. International Journal of Food Science & Technology, 52(9), 1983-1994. [ Links ]

Sperry, M. F., Silva, H. L. A., Balthazar, C. F., Esmerino, E. A., Verruck, S., Prudencio, E. S., Cucinelli, R. P. No., Tavares, M. I. B., Peixoto, J. C., Nazzaro, F., Rocha, R. S., Moraes, J., Gomes, A. S. G., Raices, R. S. L., Silva, M. C., Granato, D., Pimentel, T. C., Freitas, M. Q., & Cruz, A. G. (2018). Probiotic Minas Frescal cheese added with L. casei 01: Physicochemical and bioactivity characterization and effects on hematological/biochemical parameters of hypertensive overweighted women – A randomized double-blind pilot trial. Journal of Functional Foods, 45, 435-443. [ Links ]

Tagg, J. R., & McGiven, A. R. (1971). Assay system for bacteriocins. Applied Microbiology, 21(5), 943. PMid:4930039. [ Links ]

Temiz, H., & Çakmak, E. (2018). The effect of microbial transglutaminase on probiotic fermented milk produced using a mixture of bovine milk and soy drink. International Journal of Dairy Technology, 71(4), 906-920. [ Links ]

Thirabunyanon, M., Boonprasom, P., & Niamsup, P. (2009). Probiotic potential of lactic ccid bacteria isolated from fermented dairy milks on antiproliferation of colon cancer cells. Biotechnology Letters, 31(4), 571-576. PMid:19116692. [ Links ]

Vijayendra, S. V. N., Palanivel, G., Mahadevamma, S., & Tharanathan, R. N. (2008). Physico-chemical characterization of an exopolysaccharide produced by a non-ropy strain of Leuconostoc sp. CFR 2181 isolated from Dahi, an Indian traditional lactic fermented milk product. Carbohydrate Polymers, 72(2), 300-307. [ Links ]

Vizoso Pinto, M. G., Franz, C. M. A. P., Schillinger, U., & Holzapfel, W. H. (2006). Lactobacillus spp. with in vitro probiotic properties from human faeces and traditional fermented products. International Journal of Food Microbiology, 109(3), 205-214. PMid:16503361. [ Links ]

Yuksekdag, Z. N., & Aslım, B. (2010). Assessment of potential probiotic and starter properties of Pediococcu spp. isolated from Turkish-Type Fermented Sausages (Sucuk). Journal of Microbiology and Biotechnology, 20(1), 161-168. PMid:20134248. [ Links ]

Zhu, W. M., Liu, W., & Wu, D. Q. (2000). Isolation and characterization of a new bacteriocin from Lactobacillus gasseri KT7. Journal of Applied Microbiology, 88(5), 877-886. PMid:10792549. [ Links ]

Received: February 23, 2019; Accepted: August 24, 2019

*Corresponding author:

Creative Commons License This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.