Effectiveness Evaluation of a University-Society Engagement Project in the Mechanical Area

Evaluación de la Eficacia de un Proyecto de Vinculación con la Sociedad en el Área de Mecánica

Víctor Lenin Montaño Roldan

Universidad Técnica Luis Vargas Torres de Esmerandas, victor.montano@utelvt.edu.ec

https://orcid.org/0000-0003-4424-048X

Luis Hidalgo Solorzano

Universidad Técnica Luis Vargas Torres de Esmerandas, luis.hidalgo@utelvt.edu.ec

https://orcid.org/0000-0001-7366-9141

Martin Mateo Ramírez Márquez

Universidad Técnica Luis Vargas Torres de Esmerandas, martin.ramirez@utelvt.edu.ec

https://orcid.org/0000-0002-2559-0372

Jerson Valdez Ibarra

Universidad Técnica Luis Vargas Torres de Esmerandas, jerson.valdez@utelvt.edu.ec

https://orcid.org/0009-0002-0551-6463

Evaluating university-community engagement projects remains an unresolved challenge in Latin American higher education, and Ecuador is no exception. This research set out to do just that: go beyond the numbers to understand the true impact of the “Strengthening Technical Skills for Community Development” Project, implemented by the Mechanical Engineering Program of the Faculty of Engineering at the Luis Vargas Torres Technical University of Esmeraldas (UTLVTE) during the first semester of 2025. The initiative consisted of a technical training program in arc welding, in which 52 young people from Esmeraldas from vulnerable sectors (the Codesa sector) participated. In working with them, the university not only shared knowledge but also demonstrated a willingness to listen, trust, and commitment. Through a mixed-methods approach, quantitative tools were combined with an in-depth qualitative analysis of the participants’ perceptions. The results were compelling: 100% of the beneficiaries reported being fully satisfied with the technical content received, the facilitators’ performance during the learning process, and the practical applicability of what they learned in their daily lives. Furthermore, 80.76% of participants rated the provided materials positively, and the project achieved an overall efficiency rate of 90.47%, reflecting committed and well-directed management. Throughout the process, a profound transformation was observed in the participants’ self-esteem and outlook on life. These findings were compared with international methodological frameworks and the faculty’s prior experiences, which allowed for the validation of the process’s coherence and paved the way for future replications. The project demonstrated high effectiveness in its technical, social, and educational dimensions, establishing itself as a replicable model for new outreach interventions in the field of Engineering.

Keywords: Project evaluation, community engagement, educational effectiveness, welding, technical training, impact indicators.

Resumen

La evaluación de los proyectos de vinculación entre la universidad y la sociedad sigue siendo un desafío pendiente en la educación superior latinoamericana, y Ecuador no es la excepción. Esta investigación se propuso justamente eso: ir más allá de los números para comprender el verdadero impacto del Proyecto “Fortalecimiento de Competencias Técnicas para el Desarrollo Comunitario”, ejecutado por la Carrera de Mecánica de la Facultad de Ingenierías de la Universidad Técnica Luis Vargas Torres de Esmeraldas (UTLVTE) durante el primer semestre de 2025. La iniciativa consistió en un proceso de capacitación técnica en soldadura por arco voltaico, del que participaron 52 jóvenes esmeraldeños provenientes de sectores vulnerables (sector Codesa). Frente a ellos, la universidad no solo desplegó conocimientos, sino también escucha, confianza y compromiso. A través de un enfoque metodológico mixto, se combinaron herramientas cuantitativas, con un análisis cualitativo profundo de las percepciones de los participantes. Los resultados obtenidos fueron contundentes: el 100% de los beneficiarios manifestó estar plenamente satisfecho con el contenido técnico recibido, el desempeño de los facilitadores durante el proceso de aprendizaje y la aplicabilidad práctica de lo aprendido en su vida cotidiana. Asimismo, el 80,76% valoró positivamente los recursos materiales entregados, y el proyecto alcanzó una eficiencia global del 90,47%, lo que refleja una gestión comprometida y bien orientada. Durante el proceso, se observó una transformación profunda en la autoestima y en la proyección de vida de los participantes. Estos hallazgos fueron contrastados con marcos metodológicos internacionales y con experiencias previas del cuerpo docente, lo que permitió validar la coherencia del proceso y abrir caminos para futuras réplicas. El proyecto demostró una alta efectividad en sus dimensiones técnica, social y educativa, consolidándose como un modelo reproducible para nuevas intervenciones de vinculación en el campo de la ingeniería.

Palabras clave: Evaluación de proyectos, vinculación con la sociedad, eficacia formativa, soldadura, formación técnica, indicadores de impacto.

Introduction

Assessing the effectiveness of university-community engagement projects is a theoretical and methodological challenge for higher education today. As Wanjiru and Liu point out, “university-community engagement can take many forms and impact multiple stakeholders, and an evaluation focused solely on university perspectives could overlook the community perspective, which is equally important.” This complexity is accentuated in contexts of high social vulnerability, where technical training projects seek not only to transfer knowledge but also to bring about profound changes in the participants’ lives.

Various studies agree on a fundamental point: to truly understand what happens in these processes, we need more robust methodologies capable of capturing their complexity without oversimplifying it. Bekele et al. note that “none of the existing methodological and conceptual frameworks provides a holistic analysis of meaningful engagement indicators that transcend specific contexts of application,” which makes “collecting data on university-society engagement and drawing meaningful interpretations a challenge for researchers and professionals.” In the Ecuadorian context, this concern is shared by Jaramillo Tituaña et al., who note that “the lack of a clear and standardized methodology for evaluating engagement projects can result in a subjective and inconsistent assessment of the results obtained, which hinders informed decision-making and the continuous improvement of programs.”

In response to this need, proposals have emerged, such as that of Gendle and Tapler, who developed the Community Engagement and Partnership Inventory (CEPI), a tool that “provides a systematic evaluation of programs, applying the principles of critical global inquiry in such a way that administration and scoring methods can be flexible and adapted to the needs of the program and the institution.” Welding is a trade learned through hands-on practice and a keen eye for detail; therefore, measuring whether a training program truly works goes beyond exams. Yotee et al. demonstrated that “the visual inspection training program using the SCHEMA teaching model for industrial technicians resulted in an 81.52% increase in efficiency,” validating the importance of structured pedagogical models in this field.

“Community engagement is a function that allows universities to realign their goals and visions for the future, while remaining grounded and recognizing themselves as an integral part of society.” In this regard, Cabrera-Jara et al. point out that “instructional planning is a key element for students to learn how to learn, but for this to happen, there must be constant innovation in teaching and learning methods, as well as planning for the processes involved in teaching and putting what has been learned into practice.” They also clarify that, “today’s universities have incorporated community engagement into their curricula. This is a prerequisite for graduation, ensuring that future professionals acquire knowledge and engage with their field of work according to their specialization.”

The case of Esmeraldas, a province in Ecuador characterized by high levels of structural poverty and youth violence, illustrates the urgency of this type of intervention. The needs assessment for this project revealed that in the target community (Codesa sector), 65% of the population has only a high school education, which limits access to specialized job skills and perpetuates socioeconomic disparities. This information, gathered through a participatory process, confirmed the importance of welding training, a trade in high demand in local manufacturing industries (agriculture, construction, metallurgy).

Higher education institutions (HEIs), and in particular the “Luis Vargas Torres” Technical University of Esmeraldas (UTLVTE), have an obligation to address these realities through their core functions. The Academic Regulations of the Council for Higher Education (CES) define engagement with society in Article 4 as the function that “fosters capacity-building and the exchange of knowledge in accordance with the academic fields of the HEIs to ensure the development of effective responses to the needs and challenges of their environment.” At the institutional level, Article 7 of the University Statutes establishes as one of its objectives “to develop proposals and approaches to address the problems of the province, region, and country,” and Article 2 of the Engagement Regulations prioritizes “ongoing attention to the needs of the social environment, especially those of vulnerable sectors of the population and Historically Excluded Groups (GHE).”

This tradition of community engagement has been previously documented within the faculty itself. Viscaino Valencia et al. evaluated the performance of mechanical engineering students in a knowledge-transfer project in the Camarones parish, demonstrating that “all students achieved a score of 8.0 or higher, leading to the conclusion that their knowledge and skills in relation to their professional profile are optimal.” This background evidence shows that the School of Engineering has accumulated experience in this type of intervention and underscores the need to now evaluate not only student performance but also the project’s overall effectiveness in the community.

Therefore, the objective of this study is to evaluate the effectiveness of the outreach project “Strengthening Technical Skills for Community Development” during its first implementation period in its technical, social, and educational dimensions, proposing an integrated framework that contributes to the development of evaluation methodologies for future interventions in the field of mechanics.

Methodology

The research adopted a mixed-methods design, combining a systematic review of the project’s documentation with an analysis of quantitative and qualitative data collected during its implementation.

The target population was determined based on a participatory needs assessment conducted in March 2025, which identified the Tabiazo Parish and the Codesa sector (Youth Center) as priority intervention areas. The final sample consisted of 52 young people (out of a target population of 70), selected through simple random sampling, aged between 18 and 35, with an educational profile of 65% high school, 27% vocational, and 8% college. The sample is representative with a 95% confidence level and a margin of error of 5.8%, which is considered acceptable.

The program “Strengthening Technical Skills for Community Development,” implemented during the IS-2025 academic period (September through December 2025), was structured into three components:

a.  Outreach and Assessment: Outreach campaign and administration of needs assessments.

b.  Theoretical and Practical Training: Training in electric arc welding, with an emphasis on safety standards, welding techniques, and supervised practice.

c.  Evaluation and Closure: Administration of satisfaction surveys, statistical analysis, preparation of reports and a scientific article, and the fabrication of 50 chairs for the community.

The methodological design was inspired by previous experiences evaluating community engagement projects within the faculty itself. Viscaino Valencia et al. employed a mixed-methods approach to evaluate student performance, using a questionnaire that measured three components: knowledge, skills, and attitudes, with a weighting of 

T=0.50(K)+0.30(S)+0.20(A). Following this precedent, the present study adapted a similar approach, but one geared toward measuring the beneficiaries’ perceptions and the project’s efficiency, rather than student performance.

To construct the indicators, the recommendations of Del Cioppo and Bello were followed, who state that “impact indicators are constructed in such a way as to involve the learning process of community service projects with the participants who evaluate them.” An impact assessment survey using a 5-point Likert scale was administered, evaluating five dimensions: technical content, instructor, resources and materials, practical applicability, and overall organization. Additionally, management indicators were calculated: coverage, activity completion, and objective achievement, resulting in an overall efficiency of 90.47%.

Complementarily, participant observation was conducted during the practical sessions, along with an analysis of monthly and semiannual reports.

The questionnaire was subjected to a validation process to ensure its reliability and relevance, following the example of Jaramillo Tituaña et al., who reported that their instrument achieved “an average Content Validity Ratio (CVR) of 0.85.”

Quantitative data were processed using descriptive statistics (frequencies and percentages). Qualitative data were analyzed using thematic content analysis, identifying emerging categories related to empowerment.

Results

The results of this research demonstrate that the evaluated project constitutes a successful model of social intervention through mechanical engineering. The combination of a participatory assessment, high-quality technical training, and a community-based final product generated a multidimensional impact that goes beyond the mere transfer of knowledge, aligning with the methodological frameworks proposed by the specialized literature.

The Technical and Pedagogical Dimensions of Success

The quantitative finding of 100% satisfaction in key areas such as technical content, teaching performance, and practical applicability validates the program’s relevance and the effectiveness of the methodology employed. This result is consistent with previous welding training experiences that highlight the importance of combining theory and practice with a rigorous focus on industrial safety. Likewise, the high level of satisfaction with resources and materials (80.76% “Very satisfied”) confirms the adequate planning and execution of the project, in line with the quality management standards required by the sector.

The Consolidation of an Institutional Outreach Model

A comparison of the results of this study with those obtained by Viscaino Valencia et al. in the Camarones parish reveals a consistent pattern of success in the outreach projects of the UTLVTE School of Engineering. While the Camarones study conducted by Viscaino et al. focused on evaluating student performance during knowledge transfer in metalworking processes—documenting that 92% of participants achieved “very satisfactory” levels—the present study complements this perspective by assessing the beneficiaries’ perceptions and the project’s efficiency. Taken together, both studies demonstrate that the school has succeeded in establishing a community engagement model that is effective both for the education of its students (as demonstrated by the Camarones study) and for the empowerment of communities (as demonstrated by the present study). This two-way impact is precisely what the Academic Regulations define in Article 4 as a high-quality community engagement initiative: “it generates capacities and knowledge exchange in accordance with the academic fields of higher education institutions.”

Evaluating Community Engagement: Toward an Integrative Framework

The experience in Esmeraldas fits within the contemporary discussion on how to evaluate university community engagement. As Bekele et al. point out, “collecting data on the university’s engagement with society and drawing meaningful interpretations from it is a challenge for researchers and professionals.” In this regard, Jaramillo Tituaña et al. emphasize that “engagement with society must be a two-way street. Higher education institutions must not only transfer knowledge but also allow themselves to be influenced by society’s knowledge and demands, enabling these to impact their curricula, teaching methodologies, and lines of research.” This call for bidirectionality resonates with the findings of our study, in which interaction with the community not only benefited the 52 young people who received training but also enriched the education of the participating college students and generated knowledge applicable to future interventions.

Our study addresses this challenge through a multi-method approach that combines quantitative efficiency indicators (coverage: 71.43%; compliance: 100%) with qualitative evidence of social transformation, following the IAIA recommendations cited by Del Cioppo and Bello, who note that “Impact Assessment can help us design and implement better policies, plans, programs, and projects that will address major challenges such as climate change, biodiversity loss, population growth, urbanization, conflicts over increasingly scarce resources, inequality, and new technological opportunities.” This approach aligns with the proposal by Wanjiru and Liu, who developed a conceptual framework “consisting of three domains of university-community engagement: purpose, process, and community impacts.”

Empowerment and Psychological Well-Being

The most significant finding of this study is the qualitative dimension of empowerment. Furniture making served as a catalyst that transformed abstract learning into concrete achievement and social recognition. This impact on well-being and personal growth resonates with international research. The CWB Welding Foundation documents that “hands-on learning does more than teach technical skills. It builds confidence, reduces anxiety, and creates a sense of belonging. For many students, welding programs have been a lifeline, offering purpose and stability during difficult times.” Similarly, in the welding program at Utah State University, instructor Chloe Wilson observes that “welding changes lives here. Students prove to themselves that they can do something difficult, and that confidence carries over into everything else.”

Beyond Job Placement: Empowerment as Resilience

The relevance of this approach becomes evident when contrasted with programs focused exclusively on job placement. Byrd concluded that “all participants in welding training experienced anxiety during welding tests,” and that “a training program integrated with virtual reality will reduce anxiety better than a 100% virtual program.” By prioritizing empowerment and community recognition, our project equipped young people with resilience that protects them from frustration, building human and social capital that transcends the wait for formal employment.

Study Limitations: The main limitation is that the impact evaluation was conducted immediately after the intervention, leaving the medium-term sustainability of the empowerment unknown—a common challenge in evaluating community engagement projects in the Ecuadorian context.

Conclusions

This study evaluated the effectiveness of the community engagement project “Strengthening Technical Skills for Community Development,” implemented by UTLVTE in 2025. The findings lead to the conclusion that the program constitutes a successful model of social intervention through mechanical engineering, with an overall efficiency of 90.47% and 100% satisfaction in the key dimensions of the training.

The experience in Esmeraldas adds to a growing body of evidence generated by UTLVTE itself regarding the effectiveness of its community engagement projects. Previous studies, such as that by Viscaino et al. in the Camarones parish, had already demonstrated the faculty’s ability to transfer high-quality knowledge to students. This study complements that finding by demonstrating that such knowledge transfer also generates a profound and transformative impact on the beneficiary communities, thereby consolidating a virtuous cycle of education and development.

The main contribution of this research lies in demonstrating that evaluating the effectiveness of community engagement projects requires a multidimensional approach that integrates: (1) efficiency indicators (coverage, compliance), (2) beneficiary satisfaction assessments, and (3) qualitative evidence of social transformation and empowerment.

The experience in Esmeraldas aligns with international evidence documenting the multidimensional impact of welding training. In line with the findings of the CWB Welding Foundation and Utah State University, this project demonstrated that hands-on learning not only teaches technical skills but also builds confidence, reduces anxiety, and fosters a sense of community. The entrepreneurial potential observed among young people opens up prospects for long-term economic sustainability.

This article and the article “Youth Empowerment Through Technical Welding Training” plausibly demonstrate that the community engagement project has achieved its objectives, validating an intervention model that integrates technical excellence, social relevance, and personal empowerment, and establishing itself as a replicable example for future initiatives at higher education institutions seeking to align their core functions to contribute to social transformation in contexts of vulnerability.

Reference

Bekele, T. A., Ofoyuru, D. T., & Woldegiorgis, E. T. (2024). Assessing university-society engagements: Towards a methodological framework. 49, 2, 201–221. Innovative Higher Education. doi:https://doi.org/10.1007/s10755-024-09712-8

Byrd, A. P. (2014). Identifying the effects of human factors and training methods on a weld training program. [Tesis doctoral, Iowa State University].

Cabrera Jaramillo, L. A., & Corozo Arroyo, G. A. (2024). Propuesta de guía técnica para la capacitación en gestión de soldadura enfocada a estructuras metálicas.

Cabrera-Jara, C. A., Luy-Navarrete, W. A., & Merino Velásquez, J. (2023). Planificación universitaria y sostenibilidad de programas y/o proyectos de vinculación con la sociedad. 6 , 12, 256-267. Revista Científica. doi:https://doi.org/10.46296/rc.v6i12.0154

Consejo de Educación Superior [CES]. (2019). Reglamento de Régimen Académico (RPC-SO-08-No. 111-2019).

CWB Welding Foundation. (05 de Febrero de 2026). Building skills and resilience: How the Marinucci Family Foundation transformed welding education and student well-being across Canada. Obtenido de https://www.cwbweldingfoundation.org/building-skills-and-resilience

Del Cioppo, J., & Bello, M. (2018). Indicadores de impacto social para evaluación de proyectos de vinculación con la colectividad. Económicas CUC. 39, 1, 105-116. doi:http://dx.doi.org/10.17981/econcuc.39.1.2018.07

Gendle, M. H., & Tapler, A. S. (2024). The Community Engagement and Partnership Inventory (CEPI): An aspirational open-source instrument to assess community-based global learning programs. 30, 2. Michigan Journal of Community Service Learning. Obtenido de https://doi.org/10.3998/mjcsl.4909

Hempel, J. F., & Guidek, D. (2024). Capacitación sobre Agresores en los Procesos de Soldadura en la Educación Media. Universidad Nacional de Misiones.

Jaramillo Tituaña, B. A., & Alcívar Cedeño, A. K. (2025). Metodología para la evaluación de los proyectos de vinculación del Instituto Superior Tecnológico Tena. 7, 1, 315-334. Revista Científica Arbitrada Multidisciplinaria PENTACIENCIAS.

Mesias - Simisterra, Á., Aguilar - Mora, M., Gonzalez - Vega, A., & Reina - Tenorio, L. (2026). Empoderamiento juvenil mediante capacitación técnica en soldadura.

Total Soft Bank; Pusan National University; Korea Polytechnics. . (2024). A study on the analysis effectiveness of the virtual welding simulator for welding manpower development. KCI.

Utah State University. . (15 de Mayo de 2025). From sparks to success: Welding program grows at USU Blanding. Utah State University Today. Utah State University Today. Obtenido de https://www.usu.edu/today/story/from-sparks-to-success-welding-program-grows-at-usu-blanding

UTLVTE. (2024). Estatuto Universitario.

UTLVTE. (2024). Reglamento de Vinculación con la Sociedad de la UTLVTE - 2024.

Viscaino Valencia, P. R., Aguilar Mora, M. L., Mera Mosquera, A. R., & Montaño Roldan, V. L. (2021). Transferencia del conocimiento de mecánica en la parroquia Camarones, cantón Esmeraldas-Ecuador. 4, 2. Journal of Business and Entrepreneurial Studies. doi:https://doi.org/10.37956/jbes.v4i2.84

Wanjiru, I. R., & Liu, X. (2022). Evaluating university–community engagement through a community-based lens: What indicators are suitable? 26 , 2, 43-58. Journal of Higher Education Outreach and Engagement.

Yotee, S., Dhinnabutra, P., Khrueaphue, T., Thammasang, S., & Laoha, C. (2025). Development and evaluation of a visual welding inspection skills training package using the SCHEMA teaching model for industrial technicians. Journal of Technical and Engineer.