Experiment

To analyse the hypotheses on peculiarities and preconditions for CI development, a scientific experiment was launched alongside with the quantitative and qualitative research. As all the projects are unique, a possibility to have a control group and experimental groups with identical features was absent and, therefore, quasi-experimental research methods were invoked.

The experiment has been conducted in 2 stages.

The first stage was exploratory. The researchers used certain criteria to compile a list of online communities (the list was revised on the basis of the data collected during quantitative and qualitative interviews) and observe projects practically implemented by virtual communities. The chosen subjects were observed in accordance with the designed survey scheme (representative parameters) and the collected data underwent qualitative analysis and summarizes to make corresponding conclusions. At the onset of the experiment, the researchers conducted a natural experiment with no direct interference into activities of the researched online community in order to avoid outside influence and its effects. On completion of the initial stage, a selection of projects for further monitoring was carried out and CI criteria were adjusted.

The second stage of the experiment was an integral development of CI Potential Index. After the conceptual framework of CI Potential Index was developed, the experiment continued to empirically evaluate CI potential in selected online communities. Apart from monitoring the communities, the stage incorporated negotiations with platform developers and administrators to get access to specific web analytics data.
 

Results.

The first stage of observation revealed the complexity of monitoring online community activities. Obviously, not all aspects of performance can be measured by quantitative criteria, but some numeric data are extremely important. Measuring such data over a period could help diagnose and prevent reduction of community members' motivation or diminished activities. Testing demonstrated that some of criteria could be attributed to more than one element of the framework.
However, the unique criteria could have a different level of influence on different elements. In addition, different criteria for monitoring the unique element could be of different importance. Therefore, it would be expedient to rank each criterion by its relevance. However, researchers could not access reliable data at this stage of experiment. Therefore, the importance and correlations of diverse criteria were not analyzed yet and are planned for upcoming research stages. Moreover, the framework could be more sophisticated by demonstrating cause-effect links between criteria where applicable. However, for identification and validation of such relationships, other research techniques are required ensuring collection and analysis of actual data and testing of hypotheses.

The second stage of observation and evaluation mostly confirmed and elaborated upon the conclusions made in the initial stage of observation and evaluation. The evaluation of the creativity component confirmed the conclusions of the initial study and demonstrated that it is, indeed, difficult to gather data, especially about sex, age and nationality of users. However, researchers noticed that if it was possible to gain independent access to the Google Analytics data of the online community’s website and the initiators had enabled the presentation of demographic data and interest reports, then it was possible to acquire relatively objective data collected according to the Google method (Support.google.com, 2014) not only about the age and sex of the users (what has been analyzed in this project), but about their general interests, as well. In consideration of the difficulty of collecting data in most cases, i.e., when researchers do not have access to such data, less importance should be placed on this criterion in the future research.

The variety of issues addressed by online communities was great. It is very difficult to evaluate and compare communities based on the variety. A comparison would be more meaningful in a more homogeneous selection of online communities. Analysing such communities would provide for the possibility of applying data mining and web scraping techniques, which would improve the quality and reliability of such an analysis, especially with regard to small communities that use a separate domain for their activity. This criterion, along with the evaluation of the quality of problems, is one of the most important, and thus, more importance should be placed on it in the future research.

It was noticed that the majority of online communities use standard modules that allow the spread of information through Facebook, Twitter, Google+, LinkedIn and e-mail; however, very few communities use these platforms to the full extent. There are no elements of competition or elements of games in these communities either. However, in consideration of the missions and visions set out by the communities, not all of these tools are always necessary, thus, it would be meaningful to place less importance on these criteria in upcoming studies.

The evaluation of critical mass attraction (the “swarm effect”) is a difficult undertaking, especially in the context of such a variety of communities. However, this evaluation can be very important and great significance could be attributed to the formed criteria if the future research meets the following criteria: a) access to data from Google Analytics or a similar system is acquired for an object under observation; b) the “swarm effect” is certainly necessary for an organisation or initiative in the
implementation of its mission and vision (more is not always better); c) it is possible to identify specific supporters that create added value when solving a problem (e.g., by providing specific suggestions, synthesizing and analyzing information) and to distinguish them from the general mass of commentators, critics and “likers” on Facebook.

The analysis of effectiveness of problem-solving and degree of decentralisation and integration demonstrated the low level of maturity of almost all of the online communities when analyzing and solving problems by the collective method. With rare exceptions, exchanges of information are dominant. This correlates with the general level of passiveness in society, the level that is also demonstrated by other studies (e.g., the Lithuanian Civil Society Index). The observation was made that communities that seek to analyse problems and provide feedback as well as generalised and objective conclusions receive higher evaluations for other criteria, as well (technological training, analysis of alternatives, variety of ways to express views, procedures that ensure equal opportunity to have a say, privacy and anonymity issues). Thus, future research should pay more attention to the level of comprehensiveness of alternative analysis, and to measuring as well as analyzing the depth of problem analysis. Great significance must accordingly be attributed to the evaluation criteria of these areas.

The evaluation of self-organisation potential revealed that technological training was often superior to procedural training. That is, technical possibilities have been implemented, but there are no procedural explanations about how to use them or the final results they could lead to. The evaluation revealed a low level of descriptiveness of general norms, procedures and activities. There are also a few exceptions that do present comprehensive information, including the community values, history, terminology, a video and all the technological possibilities about how to express views and aggregate solution from them. Communities that have figured out their ideological and procedural levels are distinctly better prepared technologically speaking and are better at engaging their members. However, even in such cases, everything depends on the specific issue at hand and the additional efforts expended on the dissemination of information about the problem or idea. A great disparity was identified between the different communities in the dimension of selforganisation. Thus, communities should be grouped into two categories in future research: mature communities and developing communities. When evaluating developing communities, more significance should be attributed to criteria, such as general norms, procedures and values, whereas an analysis of mature communities would benefit from a more appropriate assessment of balance, technological and procedural accessibility and, additionally, leadership.

As in the case of assessing self-organisation, evaluation of CI emergence intensity has revealed that there is a great disparity between developing and mature communities. However, even the best ones can reach an average level. During the course of the observation, a hypothesis was formulated that the main criterion, in this case, should be the degree of creation of qualitatively new output, such as ideas, activity, structured views, competency development and other forms. The conclusion can be drawn that the formation of Collective Intelligence in online communities is in its infancy, when it is too early to speak of specific results. However, an increase in civic engagement can also be viewed as collective consciousness and, at the same time, a form of collective intellect.

The evaluation of adaptability potential demonstrates promising tendencies in the online communities of Lithuania. Unlike with evaluation of self-organisation and CI emergence, communities are successfully dealing with problems and implementing their activity with a view to fulfilling their vision and mission. By carrying out their activity, most of the communities are in the active process of learning and exchanging information and this creates the preconditions for the development of Collective Intelligence in Lithuania.