Participatory action research

Participatory action research (PAR) is an approach to action research emphasizing participation and action by members of communities affected by that research. It seeks to understand the world by trying to change it, collaboratively and following reflection. PAR emphasizes collective inquiry and experimentation grounded in experience and social history. Within a PAR process, "communities of inquiry and action evolve and address questions and issues that are significant for those who participate as co-researchers". PAR contrasts with mainstream research methods, which emphasize controlled experimentation, statistical analysis, and reproducibility of findings.

PAR practitioners make a concerted effort to integrate three basic aspects of their work: participation (life in society and democracy), action (engagement with experience and history), and research (soundness in thought and the growth of knowledge). "Action unites, organically, with research" and collective processes of self-investigation. The way each component is actually understood and the relative emphasis it receives varies nonetheless from one PAR theory and practice to another. This means that PAR is not a monolithic body of ideas and methods but rather a pluralistic orientation to knowledge making and social change.

Overview

In the UK and North America the work of Kurt Lewin[21] and the Tavistock Institute in the 1940s has been influential. However alternative traditions of PAR, begin with processes that include more bottom-up organising and popular education than were envisaged by Lewin.

PAR has multiple progenitors and resists definition. It is a broad tradition of collective self-experimentation backed up by evidential reasoning, fact-finding and learning. All formulations of PAR have in common the idea that research and action must be done 'with' people and not 'on' or 'for' people. It counters scientism by promoting the grounding of knowledge in human agency and social history (as in much of political economy). Inquiry based on PAR principles makes sense of the world through collective efforts to transform it, as opposed to simply observing and studying human behaviour and people's views about reality, in the hope that meaningful change will eventually emerge.

PAR draws on a wide range of influences, both among those with professional training and those who draw on their life experience and those of their ancestors. Many draw on the work of Paulo Freire, new thinking on adult education research, the Civil Rights Movement, South Asian social movements such as the Bhumi Sena, and key initiatives such as the Participatory Research Network created in 1978 and based in New Delhi. "It has benefited from an interdisciplinary development drawing its theoretical strength from adult education, sociology, political economy, community psychology, community development, feminist studies, critical psychology, organizational development and more". The Colombian sociologist Orlando Fals Borda and others organized the first explicitly PAR conference in Cartagena, Colombia in 1977. Based on his research with peasant groups in rural Boyaca and with other underserved groups, Fals Borda called for the 'community action' component to be incorporated into the research plans of traditionally trained researchers. His recommendations to researchers committed to the struggle for justice and greater democracy in all spheres, including the business of science, are useful for all researchers and echo the teaching from many schools of research:

"Do not monopolise your knowledge nor impose arrogantly your techniques, but respect and combine your skills with the knowledge of the researched or grassroots communities, taking them as full partners and co-researchers. Do not trust elitist versions of history and science which respond to dominant interests, but be receptive to counter-narratives and try to recapture them. Do not depend solely on your culture to interpret facts, but recover local values, traits, beliefs, and arts for action by and with the research organisations. Do not impose your own ponderous scientific style for communicating results, but diffuse and share what you have learned together with the people, in a manner that is wholly understandable and even literary and pleasant, for science should not be necessarily a mystery nor a monopoly of experts and intellectuals."

PAR can be thought of as a guiding paradigm to influence and democratize the creation of knowledge making, and ground it in real community needs and learning. Knowledge production controlled by elites can sometimes further oppress marginalized populations. PAR can be a way of overcoming the ineffectiveness and elitism of conventional schooling and science, and the negative effects of market forces and industry on the workplace, community life and sustainable livelihoods.

Fundamentally, PAR pushes against the notion that experiential distance is required for objectivity in scientific and sociological research. Instead, PAR values embodied knowledge beyond "gated communities" of scholarship, bridging academia and social movements such that research and advocacy — often thought to be mutually exclusive — become intertwined. Rather than be confined by academia, participatory settings are believed to have "social value," confronting epistemological gaps that may deepen ruts of inequality and injustice.

These principles and the ongoing evolution of PAR have had a lasting legacy in fields ranging from problem solving in the workplace to community development and sustainable livelihoods, education, public health, feminist research, civic engagement and criminal justice. It is important to note that these contributions are subject to many tensions and debates on key issues such as the role of clinical psychology, critical social thinking and the pragmatic concerns of organizational learning in PAR theory and practice. Labels used to define each approach (PAR, critical PAR, action research, psychosociology, sociotechnical analysis, etc.) reflect these tensions and point to major differences that may outweigh the similarities. While a common denominator, the combination of participation, action and research reflects the fragile unity of traditions whose diverse ideological and organizational contexts kept them separate and largely ignorant of one another for several decades.

The following review focuses on traditions that incorporate the three pillars of PAR. Closely related approaches that overlap but do not bring the three components together are left out. Applied research, for instance, is not necessarily committed to participatory principles and may be initiated and controlled mostly by experts, with the implication that 'human subjects' are not invited to play a key role in science building and the framing of the research questions. As in mainstream science, this process "regards people as sources of information, as having bits of isolated knowledge, but they are neither expected nor apparently assumed able to analyze a given social reality". PAR also differs from participatory inquiry or collaborative research, contributions to knowledge that may not involve direct engagement with transformative action and social history. PAR, in contrast, has evolved from the work of activists more concerned with empowering marginalized peoples than with generating academic knowledge for its own sake. Lastly, given its commitment to the research process, PAR overlaps but is not synonymous with action learning, action reflection learning (ARL), participatory development and community development—recognized forms of problem solving and capacity building that may be carried out with no immediate concern for research and the advancement of knowledge.

Organizational life

Action research in the workplace took its initial inspiration from Lewin's work on organizational development (and Dewey's emphasis on learning from experience). Lewin's seminal contribution involves a flexible, scientific approach to planned change that proceeds through a spiral of steps, each of which is composed of 'a circle of planning, action, and fact-finding about the result of the action', towards an organizational 'climate' of democratic leadership and responsible participation that promotes critical self-inquiry and collaborative work. These steps inform Lewin's work with basic skill training groups, T-groups where community leaders and group facilitators use feedback, problem solving, role play and cognitive aids (lectures, handouts, film) to gain insights into themselves, others and groups with a view to 'unfreezing' and changing their mindsets, attitudes and behaviours. Lewin's understanding of action-research coincides with key ideas and practices developed at the influential Tavistock Institute (created in 1947)) in the UK and National Training Laboratories (NTL) in the US. An important offshoot of Tavistock thinking and practise is the sociotechnical systems perspective on workplace dynamics, guided by the idea that greater productivity or efficiency does not hinge on improved technology alone. Improvements in organizational life call instead for the interaction and 'joint optimization' of the social and technical components of workplace activity. In this perspective, the best match between the social and technical factors of organized work lies in principles of 'responsible group autonomy' and industrial democracy, as opposed to deskilling and top-down bureaucracy guided by Taylor's scientific management and linear chain of command.

NTL played a central role in the evolution of experiential learning and the application of behavioral science to improving organizations. Process consultation, team building, conflict management, and workplace group democracy and autonomy have become recurrent themes in the prolific body of literature and practice known as organizational development (OD). As with 'action science', OD is a response to calls for planned change and 'rational social management' involving a normative human relations movement and approach to worklife in capital-dominated economies. Its principal goal is to enhance an organization's performance and the worklife experience, with the assistance of a consultant, a change agent or catalyst that helps the sponsoring organization define and solve its own problems, introduce new forms of leadership and change organizational culture and learning. Diagnostic and capacity-building activities are informed, to varying degrees, by psychology, the behavioural sciences, organizational studies, or theories of leadership and social innovation. Appreciative Inquiry (AI), for instance, is an offshoot of PAR based on positive psychology. Rigorous data gathering or fact-finding methods may be used to support the inquiry process and group thinking and planning. On the whole, however, science tends to be a means, not an end. Workplace and organizational learning interventions are first and foremost problem-based, action-oriented and client-centred.

Psychosociology

Tavistock broke new ground in other ways, by meshing general medicine and psychiatry with Freudian and Jungian psychology and the social sciences to help the British army face various human resource problems. This gave rise to a field of scholarly research and professional intervention loosely known as psychosociology, particularly influential in France (CIRFIP). Several schools of thought and 'social clinical' practise belong to this tradition, all of which are critical of the experimental and expert mindset of social psychology. Most formulations of psychosociology share with OD a commitment to the relative autonomy and active participation of individuals and groups coping with problems of self-realization and goal effectiveness within larger organizations and institutions. In addition to this humanistic and democratic agenda, psychosociology uses concepts of psychoanalytic inspiration to address interpersonal relations and the interplay between self and group. It acknowledges the role of the unconscious in social behaviour and collective representations and the inevitable expression of transference and countertransference—language and behaviour that redirect unspoken feelings and anxieties to other people or physical objects taking part in the action inquiry.

The works of Balint, Jaques, and Bion are turning points in the formative years of psychosociology. Commonly cited authors in France include Amado, Barus-Michel, Dubost, Enriquez, Lévy, Gaujelac, and Giust-Desprairies. Different schools of thought and practice include Mendel's action research framed in a 'sociopsychoanalytic' perspective and Dejours's psychodynamics of work, with its emphasis on work-induced suffering and defence mechanisms. Lapassade and Lourau's 'socianalytic' interventions focus rather on institutions viewed as systems that dismantle and recompose norms and rules of social interaction over time, a perspective that builds on the principles of institutional analysis and psychotherapy. Anzieu and Martin's work on group psychoanalysis and theory of the collective 'skin-ego' is generally considered as the most faithful to the Freudian tradition. Key differences between these schools and the methods they use stem from the weight they assign to the analyst's expertise in making sense of group behaviour and views and also the social aspects of group behaviour and affect. Another issue is the extent to which the intervention is critical of broader institutional and social systems. The use of psychoanalytic concepts and the relative weight of effort dedicated to research, training and action also vary.

Applications

Community development and sustainable livelihoods

PAR emerged in the postwar years as an important contribution to intervention and self-transformation within groups, organizations and communities. It has left a singular mark on the field of rural and community development, especially in the Global South. Tools and concepts for doing research with people, including "barefoot scientists" and grassroots "organic intellectuals" (see Gramsci), are now promoted and implemented by many international development agencies, researchers, consultants, civil society and local community organizations around the world. This has resulted in countless experiments in diagnostic assessment, scenario planning and project evaluation in areas ranging from fisheries and mining to forestry, plant breeding, agriculture, farming systems research and extension, watershed management, resource mapping, environmental conflict and natural resource management, land rights, appropriate technology, local economic development, communication, tourism, leadership for sustainability, biodiversity and climate change. This prolific literature includes the many insights and methodological creativity of participatory monitoring, participatory rural appraisal (PRA) and participatory learning and action (PLA) and all action-oriented studies of local, indigenous or traditional knowledge.

On the whole, PAR applications in these fields are committed to problem solving and adaptation to nature at the household or community level, using friendly methods of scientific thinking and experimentation adapted to support rural participation and sustainable livelihoods.

Literacy, education and youth

In education, PAR practitioners inspired by the ideas of critical pedagogy and adult education are firmly committed to the politics of emancipatory action formulated by Freire, with a focus on dialogical reflection and action as means to overcome relations of domination and subordination between oppressors and the oppressed, colonizers and the colonized. The approach implies that "the silenced are not just incidental to the curiosity of the researcher but are the masters of inquiry into the underlying causes of the events in their world". Although a researcher and a sociologist, Fals Borda also has a profound distrust of conventional academia and great confidence in popular knowledge, sentiments that have had a lasting impact on the history of PAR, particularly in the fields of development, literacy, counterhegemonic education as well as youth engagement on issues ranging from violence to criminality, racial or sexual discrimination, educational justice, healthcare and the environment. When youth are included as research partners in the PAR process, it is referred to as Youth Participatory Action Research, or YPAR.

Community-based participatory research and service-learning are a more recent attempts to reconnect academic interests with education and community development. The Global Alliance on Community-Engaged Research is a promising effort to "use knowledge and community-university partnership strategies for democratic social and environmental change and justice, particularly among the most vulnerable people and places of the world." It calls for the active involvement of community members and researchers in all phases of the action inquiry process, from defining relevant research questions and topics to designing and implementing the investigation, sharing the available resources, acknowledging community-based expertise, and making the results accessible and understandable to community members and the broader public. Service learning or education is a closely related endeavour designed to encourage students to actively apply knowledge and skills to local situations, in response to local needs and with the active involvement of community members. Many online or printed guides now show how students and faculty can engage in community-based participatory research and meet academic standards at the same time.

Collaborative research in education is community-based research where pre-university teachers are the community and scientific knowledge is built on top of teachers' own interpretation of their experience and reality, with or without immediate engagement in transformative action.

Public health

PAR has made important inroads in the field of public health, in areas such as disaster relief, community-based rehabilitation, public health genomics, accident prevention, hospital care and drug prevention.Template:Rp

Because of its link to radical democratic struggles of the Civil Rights Movement and other social movements in South Asia and Latin America (see above), PAR is seen as a threat to their authority by some established elites. An international alliance university-based participatory researchers, ICPHR, omit the word "Action", preferring the less controversial term "participatory research". Photovoice is one of the strategies used in PAR and is especially useful in the public health domain. Keeping in mind the purpose of PAR, which is to benefit communities, Photovoice allows the same to happen through the media of photography. Photovoice considers helping community issues and problems reach policy makers as its primary goal.

Occupational health and safety

Participatory programs within the workplace involve employees within all levels of a workplace organization, from management to front-line staff, in the design and implementation of health and safety interventions. Some research has shown that interventions are most successful when front-line employees have a fundamental role in designing workplace interventions. Success through participatory programs may be due to a number of factors. Such factors include a better identification of potential barriers and facilitators, a greater willingness to accept interventions than those imposed strictly from upper management, and enhanced buy-in to intervention design, resulting in greater sustainability though promotion and acceptance. When designing an intervention, employees are able to consider lifestyle and other behavioral influences into solution activities that go beyond the immediate workplace.

Feminism and gender

Feminist research and women's development theory also contributed to rethinking the role of scholarship in challenging existing regimes of power, using qualitative and interpretive methods that emphasize subjectivity and self-inquiry rather than the quantitative approach of mainstream science. As did most research in the 1970s and 1980s, PAR remained androcentric. In 1987, Patricia Maguire critiqued this male-centered participatory research, arguing that "rarely have feminist and participatory action researchers acknowledged each other with mutually important contributions to the journey." Given that PAR aims to give equitable opportunity for diverse and marginalized voices to be heard, engaging gender minorities is an integral pillar in PAR's tenants. In addition to gender minorities, PAR must consider points of intersecting oppressions individuals may experience. After Maguire published Traveling Companions: Feminism, Teaching, And Action Research, PAR began to extend toward not only feminism, but also Intersectionality through Black Feminist Thought and Critical Race Theory (CRT). Today, applying an intersectional feminist lens to PAR is crucial to recognize the social categories, such as race, class, ability, gender, and sexuality, that construct individuals' power relations and lived experiences. PAR seeks to recognize the deeply complex condition of human living. Therefore, framing PAR's qualitative study methodologies through an intersectional feminist lens mobilizes all experiences – regardless of various social categories and oppressions – as legitimate sources of knowledge.

Neurodiversity

Neurodiversity has contributed to scholarship by including neurodivergent populations within research, by asking neurodivergent adults to get involved in discussing the various stages of the scientific methodology, which allows them to provide a better understanding of the research priorities within these communities. This research can challenge the ableist structure within academia where general assumptions (e.g. neurodivergence is inferior to neurotypicality), promote neurodivergent individuals as active collaborators, thus involving them in knowledge generation and ensure the theories of human cognition include strengths and weaknesses, together with lived experiences.

Civic engagement and ICT

Novel approaches to PAR in the public sphere help scale up the engaged inquiry process beyond small group dynamics. Touraine and others thus propose a 'sociology of intervention' involving the creation of artificial spaces for movement activists and non-activists to debate issues of public concern. Citizen science is another recent move to expand the scope of PAR, to include broader 'communities of interest' and citizens committed to enhancing knowledge in particular fields. In this approach to collaborative inquiry, research is actively assisted by volunteers who form an active public or network of contributing individuals. Efforts to promote public participation in the works of science owe a lot to the revolution in information and communications technology (ICT). Web 2.0 applications support virtual community interactivity and the development of user-driven content and social media, without restricted access or controlled implementation. They extend principles of open-source governance to democratic institutions, allowing citizens to actively engage in wiki-based processes of virtual journalism, public debate and policy development. Although few and far between, experiments in open politics can thus make use of ICT and the mechanics of e-democracy to facilitate communications on a large scale, towards achieving decisions that best serve the public interest.

In the same spirit, discursive or deliberative democracy calls for public discussion, transparency and pluralism in political decision-making, lawmaking and institutional life. Fact-finding and the outputs of science are made accessible to participants and may be subject to extensive media coverage, scientific peer review, deliberative opinion polling and adversarial presentations of competing arguments and predictive claims. The methodology of Citizens' jury is interesting in this regard. It involves people selected at random from a local or national population who are provided opportunities to question 'witnesses' and collectively form a 'judgment' on the issue at hand.

ICTs, open politics and deliberative democracy usher in new strategies to engage governments, scientists, civil society organizations and interested citizens in policy-related discussions of science and technology. These trends represent an invitation to explore novel ways of doing PAR on a broader scale.

Criminal justice

Compared to other fields, PAR frameworks in criminal justice are relatively new. But growing support for community-based alternatives to the criminal justice system has sparked interest in PAR in criminological settings. Participatory action research in criminal justice includes system-impacted people themselves in research and advocacy conducted by academics or other experts. Because system-impacted people hold experiential knowledge of the conditions and practices of the justice system, they may be able to more effectively expose and articulate problems with that system. Many people who have been incarcerated are also able to share with researchers facets of the justice system that are invisible to the outside world or are difficult to understand without first-hand experience. Proponents of PAR in criminal justice believe that including those most impacted by the justice system in research is crucial because the presence of these individuals precludes the possibility of misunderstanding or compounding harms of the justice system in that research.

Participants in PAR may also hold knowledge or education in more traditional academic fields, like law, policy or government that can inform criminological research. But PAR in criminology bridges the epistemological gap between knowledge gained through academia and through lived experience, connecting research to justice reform.

Ethics

Given the often delicate power balances between researchers and participants in PAR, there have been calls for a code of ethics to guide the relationship between researchers and participants in a variety of PAR fields. Norms in research ethics involving humans include respect for the autonomy of individuals and groups to deliberate about a decision and act on it. This principle is usually expressed through the free, informed and ongoing consent of those participating in research (or those representing them in the case of persons lacking the capacity to decide). Another mainstream principle is the welfare of participants who should not be exposed to any unfavourable balance of benefits and risks with participation in research aimed at the advancement of knowledge, especially those that are serious and probable. Since privacy is a factor that contributes to people's welfare, confidentiality obtained through the collection and use of data that are anonymous (e.g. survey data) or anonymized tends to be the norm. Finally, the principle of justice—equal treatment and concern for fairness and equity—calls for measures of appropriate inclusion and mechanisms to address conflicts of interests.

While the choice of appropriate norms of ethical conduct is rarely an either/or question, PAR implies a different understanding of what consent, welfare and justice entail. For one thing the people involved are not mere 'subjects' or 'participants'. They act instead as key partners in an inquiry process that may take place outside the walls of academic or corporate science. As Canada's Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans suggests, PAR requires that the terms and conditions of the collaborative process be set out in a research agreement or protocol based on mutual understanding of the project goals and objectives between the parties, subject to preliminary discussions and negotiations. Unlike individual consent forms, these terms of reference (ToR) may acknowledge collective rights, interests and mutual obligations. While they are legalistic in their genesis, they are usually based on interpersonal relationships and a history of trust rather than the language of legal forms and contracts.

Another implication of PAR ethics is that partners must protect themselves and each other against potential risks, by mitigating the negative consequences of their collaborative work and pursuing the welfare of all parties concerned. This does not preclude battles against dominant interests. Given their commitment to social justice and transformative action, some PAR projects may be critical of existing social structures and struggle against the policies and interests of individuals, groups and institutions accountable for their actions, creating circumstances of danger. Public-facing action can also be dangerous for some marginalized populations, such as survivors of domestic violence.

In some fields of PAR it is believed that an ethics of participation should go beyond avoidance of harm. For participatory settings that engage with marginalized or oppressed populations, including criminal justice, PAR can be mobilized to actively support individuals. An "ethic of empowerment" encourages researchers to consider participants as standing on equal epistemological footing, with equal say in research decisions. Within this ethical framework, PAR doesn't just affect change in the world but also directly improves the lives of the research participants. An "ethic of empowerment" may require a systemic shift in the way researchers view and talk about oppressed communities — often as degenerate or helpless. If not practiced in a way that actively considers the knowledge of participants, PAR can become manipulative. Participatory settings in which participants are tokenized or serve only as sources of information without joint power in decision-making processes can exploit rather than empower.

By definition, PAR is always a step into the unknown, raising new questions and creating new risks over time. Given its emergent properties and responsiveness to social context and needs, PAR cannot limit discussions and decisions about ethics to the design and proposal phase. Norms of ethical conduct and their implications may have to be revisited as the project unfolds.Template:Rp This has implications, both in resources and practice, for the ability to subject the research to true ethical oversight in the way that traditional research has come to be regulated.

Challenges

PAR offers a long history of experimentation with evidence-based and people-based inquiry, a groundbreaking alternative to mainstream positive science. As with positivism, the approach creates many challenges as well as debates on what counts as participation, action and research. Differences in theoretical commitments (Lewinian, Habermasian, Freirean, psychoanalytic, feminist, etc.) and methodological inclinations (quantitative, qualitative, mixed) are numerous and profound. This is not necessarily a problem, given the pluralistic value system built into PAR. Ways to better answer questions pertaining to PAR's relationship with science and social history are nonetheless key to its future.

One critical question concerns the problem-solving orientation of engaged inquiry—the rational means-ends focus of most PAR experiments as they affect organizational performance or material livelihoods, for instance. In the clinical perspective of French psychosociology, a pragmatic orientation to inquiry neglects forms of understanding and consciousness that are not strictly instrumental and rational. PAR must pay equal attention the interconnections of self-awareness, the unconscious and life in society.

Another issue, more widely debated, is scale—how to address broad-based systems of power and issues of complexity, especially those of another development on a global scale. How can PAR develop a macro-orientation to democratic dialogue and meet challenges of the 21st Century, by joining movements to support justice and solidarity on both local and global scales? By keeping things closely tied to local group dynamics, PAR runs the risk of substituting small-scale participation for genuine democracy and fails to develop strategies for social transformation on all levels. Given its political implications, community-based action research and its consensus ethos have been known to fall prey to powerful stakeholders and serve as Trojan horses to bring global and environmental restructuring processes directly to local settings, bypassing legitimate institutional buffers and obscuring diverging interests and the exercise of power during the process. Cooptation can lead to highly manipulated outcomes. Against this criticism, others argue that, given the right circumstances, it is possible to build institutional arrangements for joint learning and action across regional and national borders that can have impacts on citizen action, national policies and global discourses.

The role of science and scholarship in PAR is another source of difference. In the Lewinian tradition, "there is nothing so practical as a good theory". Accordingly, the scientific logic of developing theory, forming and testing hypotheses, gathering measurable data and interpreting the results plays a central role. While more clinically oriented, psychosociology in France also emphasizes the distinctive role of formal research and academic work, beyond problem solving in specific contexts. Many PAR practitioners critical of mainstream science and its overemphasis on quantitative data also point out that research based on qualitative methods may be theoretically-informed and rigorous in its own way. In other traditions, however, PAR keeps great distance from both academic and corporate science. Given their emphasis on pluralism and living knowledge, many practitioners of grassroots inquiry are critical of grand theory and advanced methods for collaborative inquiry, to the point of abandoning the word "research" altogether, as in participatory action learning. Others equate research with any involvement in reflexive practice aimed at assessing problems and evaluating project or program results against group expectations. As a result, inquiry methods tend to be soft and theory remains absent or underdeveloped. Practical and theoretical efforts to overcome this ambivalence towards scholarly activity are nonetheless emerging.

See also

• Project Hayat
• Community organizing
• Cooperative inquiry
• Participatory design
• Participatory monitoring

Further reading

• Action Research, Sage, Template:ISSN
• Organizational Development Series, Addison-Wesley Business & Economics
• Educational Action Research, Template:ISSN
• International Journal of Action Research, Rainer Hampp Verlag, Template:ISSN
• Journal of Applied Behavioral Science, Sage, Template:ISSN
• Journal of Organizational Change Management
• Management Learning, Sage, Template:ISSN
• Participatory Learning and Action, IIED, Template:ISSN
• Progress in Community Health Partnerships: Research, Education, and Action, Johns Hopkins University Press, Template:ISSN
• Systems Practice and Action Research, Springer, Template:ISSN (Print) 1573-9295 (Online)